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Abstract 

The  Military  Nursing  Outcomes  Database:  Analysis  and  Expansion 

Purpose:  To  extend  MilNOD  to  additional  sites  and  to  determine  the  associations 
between  nurse  staffing  and  patient  and  nurse  outcomes. 

Design:  This  observational,  correlational  study  included  multiple  sources  of  data: 
prospectively  collected  longitudinal  staffing,  retrospectively  collected  adverse  events, 
cross-sectional  nursing  and  patient  surveys,  and  annual  pressure  ulcer  and  restraint 
prevalence  surveys. 

Methods:  The  following  indicators  were  collected  at  the  nursing  unit:  nurse  staffing, 
patient  days,  patient  turnover,  and  patient  acuity.  Patient  falls  and  nurse  medication 
administration  errors  were  extracted  from  occurrence  reports.  Nurse  needlestick  injuries 
were  obtained  from  occupational  health  or  risk  management  reports.  Pressure  ulcer 
and  restraint  data  were  collected  by  prevalence  survey  at  least  annually.  Annual  nursing 
surveys  included  education,  experience,  job  satisfaction,  and  an  evaluation  of  the 
nursing  work  environment.  Patient  surveys  included  satisfaction  with  care. 

Sample:  The  sample  includes  over  1 15,000  shifts  from  57  units  in  13  military  hospitals; 
1586  nursing  surveys;  1721  patient  satisfaction  surveys;  and  1684  pressure 
ulcer/restraint  prevalence  participants. 

Instrumentation:  The  Patient  Satisfaction  with  Nursing  Care  Questionnaire,  the  Practice 
Environment  Scale  of  the  Nursing  Work  Index,  and  a  series  of  single  item  measures 
were  used. 

Analysis:  Bayesian  hierarchical  logistic  regression  analysis  was  used  to  examine  shift 
level  staffing  associations  with  adverse  events.  Hierarchical  linear  models  were  used  to 
analyze  nurse  job  satisfaction,  patient  satisfaction,  and  work  environment  outcomes. 

Findings:  There  were  substantial  effects  of  staffing  on  adverse  events  at  the  shift  level, 
such  that  better  RN  skill  mix,  more  hours  of  care,  and  a  higher  proportion  of  civilian  staff 
resulted  in  lower  patient  and  nurse  adverse  events.  Patient  satisfaction  was  high  and 
invariant  between  hospitals.  Nurse  satisfaction  had  no  staffing  associations  but  was 
strongly  influenced  by  position. 

Implications:  The  MilNOD  project  resulted  in  a  capacity  to  collect  and  use  valid,  reliable, 
and  comparable  guality  indicator  data  to  advance  the  potential  for  patient  outcome 
benchmarking  and  evidence-based  decision  support. 
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Introduction 

Today’s  health  care  system  is  largely  the  product  of  payment  reform  and  redesign 
efforts  of  the  past  20  years.  In  the  1980s,  because  of  prospective  payment  policies, 
patients  were  discharged  from  hospitals  “sicker  and  quicker”,  requiring  nurses  to  be 
exceptionally  competent  to  manage  the  needs  of  highly  complex  patients  in  a 
compressed  time  period.  The  turbulence  and  chaos  in  health  care  escalated  in  the 
1990s  as  restructuring  efforts  changed  the  composition  of  the  hospital  workforce  by 
reducing  nursing  staff  despite  the  heightened  patient  acuity  in  all  care  settings  (Aiken, 
Clarke  &  Sloane,  2000;  Aiken  &  Fagin,  1997;  Aiken,  Sochalski  &  Lake,  1997; 

Committee  on  Quality  of  Health  Care  in  America,  2001;  Curran  &  Mazzie;  1995;  Kohn, 
Corrigan,  &  Donaldson,  1999;  Shindul-Rothschild,  Berry  &  Long-Middleton,  1996; 
Tillman,  Salyer,  Corley  &  Mark,  1997;  Walston,  Burns,  &  Kimberly,  2000;  Wiener,  2000; 
Wunderlich,  Sloan  &  Davis,  1996). 

Whereas  cost  was  the  prevailing  issue  in  health  care  in  the  past,  quality  has  now  moved 
into  the  foreground.  Critical  examinations  of  health  care  quality  commenced  with  the 
release  of  the  report  from  the  President’s  Advisory  Commission  on  Consumer 
Protection  and  Quality  in  the  Health  Care  Industry  (1998).  Shortly  thereafter,  the 
Institute  of  Medicine’s  (IOM)  report  on  patient  safety  (Kohn,  Corrigan  &  Donaldson, 
2000)  catapulted  quality  issues  into  prominence  as  the  number  one  national  health  care 
concern.  Deficiencies  in  patient  safety  issues  in  particular  and  quality  care  in  general 
were  being  exposed  at  the  same  time  that  workforce  issues  in  several  health 
professions  were  emerging  (Aiken  et  al. ,  2001;  Buerhaus  &  Staiger,  1999;  Committee 
on  Quality  of  Health  Care  in  America,  2001 ;  Bates  et  al.,  1997).  These  workforce 
issues  suggest  there  may  be  serious  and  protracted,  perhaps  even  irreversible, 
consequences  of  staffing  shortages  and  work  environment  problems  that  may  further 
compromise  the  quality  of  care  and  patient  safety  (Hinshaw  &  McClure,  2001 ).  Since 
that  time,  subsequent  reports  have  called  attention  to  the  work  environment  of  hospital 
nurses  as  being  another  source  of  patient  safety  and  quality  care  concerns  (Patrician, 
Shang,  &  Lake,  2010;  Page,  2004). 

Nurses  have  been  called  the  “backbone  of  the  health  industry”  (Altman,  1971 ,  p.  1 ).  For 
acute,  inpatient  care,  it  is  accepted  that  patients  are  admitted  to  hospitals  for  the 
purpose  of  receiving  nursing  care.  It  is  therefore  not  surprising  that  nurses  are  viewed 
as  a  safety  net  for  the  health  care  system  by  virtue  of  their  constant  presence  and 
proximity  to  patients  where  a  significant  number  of  preventable  errors  occur  (Foley, 

1 999).  If  there  is  a  gap  in  quality,  nurses  are  at  the  patient’s  side  to  catch  problems  and 
intervene  before  mistakes  happen.  Hence,  nurses  are  the  last  line  of  defense  before 
system  errors  reach  the  patient. 

Although  it  is  disconcerting,  it  is  not  entirely  surprising  that  inflammatory  media 
allegations  point  the  finger  of  blame  at  nurses  for  compromises  in  patient  safety 
(Berens,  2000).  Among  the  many  aspects  surrounding  patient  safety  that  the  press 
failed  to  note  is  one  articulated  by  Wakefield  (2001 ) — inadequate  nurse  staffing  places 
patient  care  in  jeopardy.  But  nurses  alone  cannot  be  held  accountable  when  those  who 
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establish  policies  and  make  decisions  fail  to  consider  that  staffing  may  improve  or 
compromise  patient  safety. 

The  absence  of  data  for  decision-making  feeds  the  cycle  of  targeting  nursing  for  further 
reductions  when  cost  containment  is  necessary  and  then  for  holding  nursing 
accountable  when  patient  safety  and  quality  dip  below  the  level  of  acceptability.  The 
absence  of  these  data  is  no  longer  tenable.  Such  data  are  central  to  strategic  planning, 
policy  decisions,  financial  stability,  as  well  as  patient  safety  and  quality.  “It  is  ironic  that 
hospitals  have  long  entrusted  major  portions  of  their  budgets  to  nurse  managers,  yet 
have  provided  few  tools  ...  for  ensuring  that  the  core  business  of  the  institution — 
nursing — was  being  well-managed”  (Diers,  Weaver,  Bozzo,  Allegretto,  &  Pollack,  1998, 
p.  108).  It  is  time  to  reverse  this  irony.  And  that  was  the  goal  of  this  project — to  create 
a  database  with  valid  and  reliable  nursing  data  that  will  ultimately  support  the  serious 
and  appropriate  appraisal  of  staffing  effectiveness  and  nursing’s  contribution  to  patient 
safety  and  quality  care. 

The  Military  Nursing  Workforce:  An  Army  Snapshot 

The  military  nursing  workforce  is  a  combination  of  active  duty,  reserve,  career  civilian 
and  contract  nurses.  Because  of  nursing  roles  in  support  of  the  readiness  mission,  this 
workforce  is  also  a  blend  of  Registered  Nurses  (RNs),  Licensed  Practical  Nurses 
(LPNs),  and  unlicensed  assistive  personnel,  such  as  nursing  assistants,  combat 
medics,  corpsmen,  and  technicians.  The  composition  of  the  military  active  duty  and 
reserve  nursing  workforce  is  prescribed  by  regulations.  It  represents  an  important 
distinction  from  the  civilian  workforce.  This  military  unique  feature  must  be  taken  into 
account  when  examining  patient  safety  and  nurse  staffing  effectiveness.  Therefore, 
civilian  staffing  and  outcomes  studies  may  not  be  representative  of  the  military 
structure. 

Historical  Army  Medical  Department  (AMEDD)  personnel  inventory  data  show  a  decline 
of  1 ,400  in  the  number  of  active  duty  Army  Nurse  Corps  (ANC)  officers  from  1991  to 
2001  (COL  Carol  Huff,  personal  communication,  February  22,  2002).  This  decrease  in 
the  ANC  was  part  of  a  much  larger  Department  of  Defense  effort  to  reduce  force 
structure,  affecting  all  the  Services,  including  the  Army  and  the  AMEDD.  Recently  the 
Air  Force  and  Navy  Nurse  Corps  have  come  under  order  to  reduce  in  size. 

Prior  to  the  1990s,  staffing  levels  in  military  hospitals  were  somewhat  resilient  to 
fluctuations  in  the  civilian  nurse  workforce  because  military  nurses  comprised  the 
majority  of  the  inpatient  staff.  This  is  no  longer  the  case.  Data  from  one  Army  MTF 
indicate  a  reversal  in  the  RN  workforce  composition  between  1996  and  2002.  In  1996, 
the  RN  staff  comprised  65%  ANC  officers  and  35%  civilians.  These  percentages  were 
reversed  by  2002  with  an  RN  staff  composition  of  36%  ANCs  and  64%  civilians.  A 
transition  in  the  LPN  workforce  also  occurred  during  this  time  period.  In  1996,  the  LPN 
staff  comprised  64%  Army  personnel  and  36%  were  civilians.  By  2002,  the  split  was 
more  equal  with  48%  Army  LPNs  and  52%  civilian  LPNs.  Similar  changes  are  found 
when  unlicensed  personnel  are  considered.  In  total,  military  nursing  personnel 
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accounted  for  a  70%  majority  of  the  workforce  in  1996.  Currently  military  personnel 
comprise  only  about  40%  of  the  nursing  workforce  (Patrician  et  al. ,  2011).  The 
collective  effect  of  these  shifts  resulted  in  the  requirement  for  new  civilian  nursing 
positions  military  medical  centers,  yet  difficulty  in  filling  the  positions,  especially  in  urban 
areas  such  as  Washington,  DC  and  Tacoma,  WA. 

Currently,  civilian  and  military  hospitals  across  the  country  are  experiencing  a  respite 
from  the  recent  and  looming  nursing  shortages.  This  respite  is  attributed  to  the  recent 
economic  downturn,  but  is  not  expected  to  continue  once  the  economy  rebounds 
(Buerhaus,  Staiger,  &  Auerbach,  2009).  The  increased  average  age  of  the  RN 
workforce,  upcoming  planned  retirements,  and  the  aging  population  will  all  increase 
demands  for  RNs.  According  to  Buerhaus  et  al.  (2000),  the  U.S.  will  experience  a  20% 
shortage  in  the  number  of  nurses  needed  in  our  nation’s  health  care  system  by  the  year 
2020.  This  translates  into  a  shortage  of  more  than  400,000  RNs  nationwide  (Buerhaus, 
Staiger,  &  Auberbach,  2000).  This  is  compounded  by  the  recently  documented  nursing 
faculty  shortage  and  nursing  schools  have  been  for  the  past  five  years,  turning  away 
qualified  applicants  because  of  insufficient  faculty  to  teach  them  (Allen,  2008).  The 
American  Association  of  Colleges  of  Nursing  (AACN,  2010)  reports  that  nearly  55,000 
qualified  nursing  student  applicants  are  turned  away  annually  because  of  the  nursing 
faculty  shortage.  This  alone  will  have  a  substantial  impact  on  the  pipeline  for  nurses  to 
enter  the  military. 

The  deployment  of  active  duty  nurses  and  the  subsequent  activation  of  reserve  nurses 
is  another  unique  aspect  of  the  military  nursing  workforce.  Humanitarian  and  wartime 
missions  require  military  nurses  to  leave  their  peacetime  duty  assignments  in  order  to 
provide  nursing  services  elsewhere.  Reserve  nurses  are  usually  designated  to  replace 
deployed  active  duty  nurses.  Reserve  nurses  are  trained  to  support  the  mission  of  their 
service  (Army,  Navy  or  Air  Force)  as  part  of  their  monthly  and/or  yearly  drills,  however, 
they  may  or  may  not  be  practicing  nurses  in  their  civilian  jobs.  Even  those  who  are 
practicing  nurses  may  not  work  in  the  specialty  they  are  assigned  to  in  their  reserve 
unit — e.g.  a  reservist  who  is  civilian  pediatric  nurse  may  be  assigned  to  an  adult  medical 
nurse  position.  When  deployments  occur  and  reserve  backfill  is  required,  an  orientation 
and  train  up  period  is  required  as  reservist’s  transition  for  their  new  role  in  a  new  work 
environment.  Consequently,  depending  on  the  size  of  the  MTF,  as  many  as  a  few 
hundred  trained  and  experienced  nurses  may  leave  the  patient’s  bedside  one  day  and 
be  replaced  by  nurses  who  are  transitioning  to  new  assignments  the  next  day.  To  date, 
the  impact  of  this  military-unique  aspect  of  nursing  is  not  known. 

Staffing  Effectiveness  and  Patient  Outcome  Research 

Evidence,  albeit  inconsistent  in  many  cases,  suggests  that  better  staffing  is 
associated  with  positive  patient  outcomes  (Kane,  Shamliyan,  Mueller,  Duval,  &  Wilt, 
2007;  Lang,  Hodge,  Olson,  Romano,  &  Kravitz,  2004;  Page,  2004).  Higher  nurse  to 
patient  ratios,  higher  proportions  of  registered  nurses,  and  more  total  nursing  care  hours 
have  been  linked  to  lower  patient  mortality  (Aiken,  Clarke,  Sloane,  Sochalski,  Silber, 
2002;  Aiken  et  al.,  2001 ;  Aiken,  Smith,  &  Lake,  1994;  Kane  et  al.;  Lang,  et  al.; 
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Needleman,  Buerhaus,  Meattke,  Stewart,  &  Zelevinsky,  2002),  decreased  length  of  stay 
and  a  lower  likelihood  of  patient  complications  such  as  nosocomial  infections  and 
pressure  ulcers  (Kovner  &  Gergen,  1998;  Blegen,  Goode,  &  Reed,  1998;  Lang  et  al; 
Needleman  et  al.). 

Although  research  has  amplified  the  importance  of  an  adequate  number  and  mix  of 
nurses  in  providing  high  quality  patient  care,  there  are  well  known  limitations  of  nurse 
staffing  research,  such  as  suitability  of  data  sources  for  both  staffing  and  outcome 
measures  (Clarke  &  Donaldson,  2008).  The  available  research  is  not  easily  translated 
into  managing  staffing  or  patient  outcomes  within  facilities.  For  example,  because  a 
wide  variety  of  factors  must  be  considered  in  staffing  decisions,  there  are  no  definitive 
formulas  available  to  prescribe  a  certain  number  of  nurses  or  skill  level  of  nurses  for  a 
given  unit.  Therefore,  there  has  been  a  growing  national  trend  toward  standardized 
measurements  of  nurse  staffing  and  patient  outcomes,  promulgated  by  both  the  ANA 
and  CalNOC,  to  create  nursing  services  “scorecards”  that  enable  nursing  leaders  to 
look  within  organizations  as  well  as  to  compare  their  facility  to  other  like  organizations 
(Firth,  Anderson,  &  Sewall,  2010).  Currently,  over  1,400  hospitals  participate  in  the 
ANA’s  National  Database  of  Nursing  Quality  Indicators  (NDNQI;  NDNQI,  n.d.),  and  over 
200  participate  in  the  Collaborative  Alliance  for  Nursing  Outcomes  (CalNOC;  CALNOC, 
2010). 

Consequently,  while  findings  are  beginning  to  emerge  regarding  relationships  between 
staffing  and  patient  outcomes,  a  number  of  measurement  and  analytic  issues  remain  to 
be  resolved.  Congruent  with  the  American  Nurses  Association  (ANA,  1995),  the 
National  Database  for  Nursing  Quality  Indicators  (NDNQI,  2002),  and  the  CALINOC) 
(Brown,  Donaldson,  Aydin  &  Carlson,  2001),  investigators  for  this  proposed  study 
believe  the  opportunity  to  advance  measurement  precision  lies  in  our  ability  to  capture 
nurse  staffing  and  measures  of  clinical  workload,  along  with  patient  care  outcomes  daily 
at  the  patient’s  bedside  (Donaldson,  Brown,  Bolton,  Aydin  &  Paul,  2001). 

The  MilNOD  allowed  military  nurse  leaders  and  military  nurse  researchers  to  trace  and 
analyze  daily  variation  in  staffing  with  previously  unrealized  but  essential  precision  and 
examine  its  effect  on  patient  safety  and  outcomes.  This  approach  is  supported  by 
Mitchell  and  Shortell  (1997)  who  advocated  addressing  such  questions  at  a  smaller 
aggregate  level — the  unit  instead  of  the  hospital.  Whitman  et  al.  (2002)  suggest  that 
most  hospital  systems  use  either  data  from  the  department  or  the  patient  care  unit  level 
when  reporting  outcomes  because  it  is  theses  operational  groups  who  assume  ultimate 
responsibility  for  these  outcomes.  Similarly,  the  Needleman  group  (2001 )  noted  “we 
need  to  better  understand  the  factors  influencing  both  staffing  levels  and  mix  of 
personnel  in  hospitals”  (p.  143). 

Use  of  Database  Performance  Information  for  Quality  Improvement 

Currently  U.S.  health  care  industry  efforts  focus  on  identifying  and  standardizing 
nomenclature,  integrating  nursing  data  within  patient  safety  and  patient  outcomes  data, 
and  developing  databases  and  decision  support  systems  for  nursing.  The  term 
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database  is  defined  as  a  collection  of  interrelated  files  with  records  organized  and 
stored  together  in  a  computer  system  (American  Nurses  Association,  1994).  Uses  for 
databases  include  information  retrieval,  data  sharing  among  users,  statistical  analysis, 
and  knowledge  building  (Chowdhury,  Linnarsson,  Wallgren,  Wallgren,  Wigertz,  1990; 
Graves  &  Corcoran,  1988;  Wu,  Crosby,  Ventura,  &  Finnick,  1994).  Although  the 
majority  of  existing  health  care  data  sources  are  rich  repositories  of  administrative  data, 
they  are  much  weaker  in  respect  to  clinical  data  (Jennings  &  Staggers,  1997; 

Hierholzer,  1991).  Hence,  the  availability  of  comprehensive  and  integrated  clinical 
databases  remains  scarce  (Jennings  &  Staggers,  1999). 

The  absences  of  high  quality,  retrievable  data  to  guide  cost  cutting  decisions  feeds  the 
cycle  of  targeting  nursing,  the  largest  personnel  pool  in  inpatient  facilities,  when  cost 
containment  is  necessary.  Reductions  in  nurse  staffing  have  reached  crisis  proportions 
nationally  leading  to  attempts  to  legislate  staffing  ratios  to  preserve  patient  safety  and 
quality  care  (Bolton,  et  al. ,  2001 ;  Buerhaus  &  Needleman,  2000;  Sovie  &  Jawad,  2001 ; 
Spetz,  2001 ).  Concurrent  with  staffing  reductions  is  the  loss  of  individuals  interested  in 
nursing — both  those  currently  in  the  profession  who  are  either  aging  out  or  dissatisfied 
with  their  work  environments,  as  well  as  a  severely  restricted  inflow  of  nurses  from  the 
educational  settings  (Buerhaus,  Staiger,  &  Auerbach,  2000).  Reversing  these  trends 
depends,  in  part,  on  having  better  databases  that  have  sufficient  scientific  integrity  to 
allow  for  analysis  of  patient  safety  and  quality  care  data.  These  databases  must  also 
contain  nurse  sensitive  patient  outcome  and  staffing  data. 

All  accredited  health  care  organizations  use  performance  measures  for  quality 
improvement,  but  the  degree  and  sophistication  of  use  varies.  Ideally,  performance 
measures  would  be  used  to  target  quality-improvement  initiatives,  set  goals,  identify  the 
root  cause  of  problems,  and  monitor  progress.  The  most  useful  measures  were 
standardized,  timely,  stable,  capable  of  trending,  measured  at  the  appropriate  unit  of 
analysis,  affordable  and  cost  effective,  and  relevant  (Scanlon,  Darby,  Rolph  &  Doty, 

2001 ).  The  MilNOD  meets  all  six  of  these  criteria  as  depicted  in  Table  1 . 

Table  1 

Characteristics  of  the  MilNOD 

CRITERIA _ EXPLANATION _ 

The  MilNOD  used  an  established  fixed  set  of  indicator  definitions. 
Standardized  These  definitions  are  consistent  with  those  used  by  the  NDNQI  and 
_ the  CalNOC  thus  promoting  standardization  of  nursing-specific  data. 


Timely 

The  MilNOD  acknowledged  the  need  to  make  decisions  based  on 
current  data.  The  MilNOD  provided  current  data  decision-making  via 
Quarterly  reporting  to  nursing  leaders  at  each  participating  MT 

Stable  &  Capable 
of  Trending 

For  military  nursing,  was  not  possible  to  assess  whether  patient  safety 
and  staffing  effectiveness  were  improving  or  deteriorating.  The 

MilNOD  provided  stable  measures  that  were  examined  over  time.. 

Appropriate  Unit 

The  appropriate  unit  of  analysis  is  a  key  element  of  whether  measures 
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of  Analysis 

are  actionable.  MilNOD  staffing  data  were  collected  every  shift.  All 
applicable  indicators  were  reported  to  Chief  Nurses  quarterly. 

The  MilNOD  was  developed  with  careful  consideration  of  affordable 
and  cost  effective  measures.  It  standardized  and  improved  data 

Affordable  and 
Cost  Effective 

collection  efforts  at  MTFs  and  provided  participating  MTFs  with  more 
sophisticated  data  analysis  tools  as  well  as  comparison,  target  and 
benchmarking  rates.  Resources  for  MilNOD  were  centralized,  and 
used  throughout  the  study  hospitals.  Sharing  of  protocols  and  other 
documents  were  encouraged  and  highlighted  in  the  newsletter,  The 
MilNOD  Messenger. 

Relevant 

Using  measures  that  are  specific  to  nursing,  sensitive  to  changes  in 
nursing  care  quality,  and  heavily  supported  in  the  civilian  nursing 
community  lend  credibility  to  the  relevance  of  the  MilNOD.  The 

MilNOD  collected  data  that  provided  a  better  picture  of  the  military 
nursing  workforce. 

Nurses’  Work  Environment 

In  hospitals,  where  nursing  care  remains  the  primary  intervention,  nurses  serve  as  the 
patients’  surveillance  system  (Aiken,  Sochalski,  &  Lake,  1997).  One  aspect  of  inpatient  care 
involves  assessing  patients  for  subtle  changes  that  might  indicate  the  onset  of  life- 
threatening  complications.  In  order  to  appropriately  intervene  in  such  events,  nurses  must 
have  the  autonomy  to  put  into  practice  what  they  know,  have  the  necessary  control  over 
resources  in  order  to  intervene  appropriately,  and  have  positive  relationships  with  physicians 
in  order  to  mobilize  those  resources.  The  environment  in  which  nurses  practice  is  emerging 
as  an  important  contextual  indicator  reflecting  attributes  of  the  hospital  care  setting  in  which 
nursing  services  are  provided 

Lake  (2002)  defines  the  nursing  practice  environment  as  the  conditions  under  which 
nurses  practice  that  may  contribute  to  or  detract  from  professional  nursing  practice. 
Research  into  the  work  environment  of  nurses  has  provided  ample  evidence  that  those 
with  characteristics  suggestive  of  professional  nursing  practice  are  associated  with  both 
better  nurse  staffing  and  better  patient  outcomes  (Kazanjian,  Green,  Wong,  &  Reid, 
2005).  Favorable  work  environments  for  nurses  have  also  been  associated  with  low 
levels  of  nurse  burnout,  higher  job  satisfaction,  less  turnover  and  more  positive  patient 
outcomes,  to  include  lower  mortality  and  higher  satisfaction  (Aiken,  Havens,  &  Sloane, 
2000;  Aiken  &  Sloane,  1997;  Aiken,  Sloane,  Lake,  Sochalski,  &  Weber,  1999;  Aiken, 
Smith,  &  Lake,  1994;  Brady-Schwartz,  2005;  Friese,  2005;  Kazanjian  et  al.). 

Nurses  working  in  Army  hospitals  may  differ  in  how  they  perceive  their  work 
environments,  career  options,  and  decisions  to  terminate  employment  because  of  the 
particular  nature  of  their  employment  as  well  as  demographic  peculiarities.  Army 
nurses  have  responsibilities  not  only  to  maintain  their  clinical  competency,  but  also  to 
maintain  military  skills,  such  as  weapons  firing  and  physical  fitness.  As  military  officers, 
they  are  expected  to  advance  in  leadership  education,  skills  and  positions  throughout 
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their  careers.  Many  DoD  civilian  nurses  are  also  officers  in  the  Army  Reserve,  and 
have  similar  expectations.  Army  hospitals  employ  a  mixture  of  RNs,  licensed  practical 
nurses,  and  nursing  assistants  who  are  military,  civilian,  or  contract  (agency)  nurses. 

The  highly  structured,  bureaucratic  environments  and  demands  of  military  life  might 
hinder  the  flexibility  and  stability  that  characterize  good  work  environments.  Despite  the 
possible  burdens,  there  are  great  opportunities  for  advancement  in  military  nursing. 
These  include  educational  benefits,  such  as  returning  to  school  full  time  for  advanced 
degrees  —  with  full  pay,  benefits,  and  tuition.  The  Army  Nurse  Corps  (ANC)  has  more 
ethnic  diversity  and  more  males  in  its  ranks  than  nursing  in  the  civilian  sector.  In 
addition,  the  military  rank  structure  might  facilitate  positive  working  relationships 
between  nurses  and  physicians  than  in  civilian  settings  (Patrician,  Shang,  &  Lake, 

2010). 

Studies  exploring  the  culture  and  dynamics  of  the  nursing  practice  environment  within 
the  military  health  care  system  are  dated,  sparse,  and  inconsistent  in  their  findings. 
Studies  from  the  1990s  found  poor  nurse-physician  communication  and  lack  of 
autonomy  in  military  nursing  practice  environments  (Anderson,  Maloney,  Oliver,  Brown, 
&  Hardy,  1996;  Maloney,  Anderson,  Gladd,  Brown,  &  Hardy,  1996).  However,  Foley 
and  colleagues  (2002)  found  practice  environments  in  two  Army  hospitals  characterized 
by  autonomy,  control  over  practice,  clinical  expertise  among  the  nursing  staff,  and 
collegiality  with  physicians. 

The  first  system-wide  investigation  of  the  nursing  practice  environments  within  Army 
hospitals  and  their  effect  on  nurses’  job  satisfaction,  emotional  exhaustion,  intent  to 
leave,  and  ratings  of  care  quality  found  that  the  nursing  practice  environment  had  a 
substantial  positive  association  with  all  outcomes  (Patrician,  Shang,  &  Lake,  2010).  The 
largest  effect  was  seen  for  emotional  exhaustion.  This  study  found  that  the  professional 
practice  environments  within  Army  Medical  Department  hospitals  were  characterized  as 
favorable  overall,  with  nurses  scoring  on  average  somewhere  between  magnet  and 
non-magnet  hospitals  on  the  Practice  Environment  Scale. 

Although  one  study  found  a  relationship  between  professional  practice  environments 
and  nurse-reported  patient  safety  climate  (Armstrong  &  Laschinger,  2006),  absent  from 
this  body  of  literature  are  investigations  of  the  associations  between  the  practice 
environment  and  patient  safety  outcomes,  such  as  falls  and  medical  errors,  as  well  as 
studies  on  the  moderating  effects  of  the  work  environment  on  structural  attributed  (e.g., 
staffing)  known  to  affect  patient  outcomes.. 

Summary 

The  research  agenda  for  this  multi-staged  research  program  evolved  from  the 
beliefs  and  problem  statements  noted  below. 

Global  Factors: 

•  Nursing  care  is  a  key  factor  in  the  outcomes  of  hospitalized  patients. 
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•  Additional  factors  affecting  patient  outcomes  include  severity  of  the  patient’s 
condition,  other  patient  characteristics,  services  rendered  by  other  disciplines, 
and  the  nurses’  work  environment. 

•  Systematic  research  addressing  staffing  effectiveness  and  patient  outcomes  has 
been  conducted  but  suffers  from  several  shortcomings  in  regard  to  the 
relationship  of  nursing  care  to  patient  outcomes  (Blegen,  Goode,  &  Reed,  1998). 

•  Hospital  generated,  direct  care,  staffing  data  are  the  “gold  standard”  for  use 
when  studying  staffing  effectiveness  and  patient  safety  in  the  Military  Health 
System  (MHS). 

Nurse  Staffing  Factors: 

•  Military  nursing  leaders  are  concerned  with  staffing  effectiveness  and  patient 
safety. 

•  Concerns  about  the  adequacy  of  nurse  staffing  have  heightened  as  the  nurse 
shortage  has  compromised  the  ability  of  military  hospitals  to  recruit  and  retain 
staff  to  meet  their  minimum  staffing  requirements. 

•  Having  an  adequate  number  of  nurses  at  the  bedside  to  care  for  patients  is  vital 
to  ensure  patient  safety  although  there  is  little  empirical  evidence  that  can  be 
used  by  leaders  to  determine  nurse  to  patient  staffing  ratios  (Bolton  et  al.,  2001 ; 
Buerhaus  &  Needleman,  2000;  Clarke  &  Donaldson,  2008;  Kane,  Shamlian, 
Mueller,  Duval,  &  Wilt,  2007;  Lang,  Hodge,  Olson,  Romano,  &  Kravitz,  2004; 
Sovie  &  Jaward,  2001 ). 

Nurse  Executive  Factors: 

•  Military  nursing  leaders  have  experienced  an  increased  burden  of  data  collection 
yet  they  lack  the  information  distilled  from  such  data  to  make  meaningful 
decisions  about  staffing  allocation 

•  There  is  lack  of  outcome  measurement  and  reporting  systems  in  the  MHS 
especially  those  with  outcomes  sensitive  to  nursing  care. 

•  Because  nurse-sensitive  military  outcome  data  reporting  systems  do  not  exist, 
each  request  for  nursing  care  structure  and  patient  outcome  data  requires  an 
individual  data  collection  effort. 

•  The  quality  of  data  collected  from  these  efforts  is  often  lacking.  High  quality  data 
are  needed  for  decision  making. 

•  Across  the  MHS  nursing  leaders  are  grappling  with  similar  issues  related  to 
collecting,  analyzing  and  interpreting  data  to  be  used  for  decision  making. 

•  It  is  critical  for  military  nurses  to  standardize  data  collection  processes  to 
decrease  duplication,  increase  benchmarking,  and  maximize  uniformity. 

MilNOD  Opportunities  and  Challenges: 

The  MilNOD  was  created  to  combine  the  ‘real  world’  of  hospital  data  collection  and  the 
data  needs  of  nursing  leaders  with  the  scientific  integrity  of  a  research  database  that 
meets  the  requirements  for  scientific  inquiry  (Brown,  Donaldson,  Aydin,  &  Carlson, 
2001). 
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•  This  database  relied  on  input  of  data  at  the  hospital  level;  therefore,  processes 
had  to  be  emplaced  to  ensure  data  integrity.  From  the  perspective  of  the 
hospital,  these  processes  needed  to  be  easy  to  understand  and  implement  with 
limited  hospital  resources.  From  the  perspective  of  the  researcher,  processes 
were  standardized  and  implemented  as  consistently  as  possible  across 
institutions  and  over  time. 

•  Database  quality  is  vital.  Efforts  were  applied  to  maximize  data  quality  before  the 
data  were  used  for  research  or  decision-making. 

•  Populating  a  database  for  research  purposes  alone  did  not  entice  nursing 
leaders  to  participate  in  a  nursing  outcomes  database.  Nursing  leaders  and 
hospital  commanders  saw  a  benefit  from  participating.  Reports  of  hospital 
performance  on  each  indicator  as  well  as  comparison  data  from  other 
participating  hospitals  were  a  motivating  benefit  to  nursing  leaders. 

•  A  standardized  nursing  outcomes  database  was  created  to  meet  the  above 
mentioned  requirements  and  was  used  at  MTFs  of  all  sizes  and  by  all  services  to 
benefit  military  nursing  leaders  and  their  MTFs  as  well  as  provide  valuable  data 
to  address  important  military  nursing  research  questions. 

The  MilNOD  was  developed  based  on  these  beliefs  and  tenets.  Although  many 
of  the  assertions  outlined  above  are  echoed  in  the  civilian  nursing  community,  important 
differences  between  the  provision  of  health  care  in  civilian  institutions  and  the  MHS 
necessitate  a  focused  inquiry  into  staffing  effectiveness  and  patient  safety  in  military 
hospitals.  For  example,  specific  to  nursing,  in  the  MHS: 

•  Nurse  staffing  models  included  fewer  RNs  and  more  LPNs,  medics,  corpsman  and 
nursing  assistants  due  to  military  readiness  and  force  structure  requirements 
peculiar  to  the  military  mission. 

•  Military  activities  required  nursing  personnel  to  be  away  from  the  patient  care  unit 
to  which  they  are  assigned  (from  1  hour  to  many  months). 

•  Reserve  military  nurses  were  used  to  replace  deployed  active  duty  staff  nurses. 

•  Staff  members  have  dual  roles  as  both  nurse  and  soldier. 

•  One  half  to  two-thirds  of  the  military  nursing  staff  geographically  rotate  from  MTF  to 
MTF  every  2  to  5  years;  additionally,  civilian  nurses  married  to  military  personnel 
also  rotate  frequently. 

•  More  military  RNs  have  bachelors,  masters  and  doctoral  degrees  than  their  civilian 
counterparts  (Patrician,  Shang,  &  Lake,  2010;  US  Department  of  Health  and 
Human  Services  [DHHS],  2010) 

•  More  nurses  associated  with  the  military  have  advanced  and  specialty  training  (i.e. 
ICU,  OR,  leadership)  as  compared  to  their  civilian  counterparts  (DHHS,  2010). 

•  More  new  graduate  nurses  take  care  of  patients  at  the  bedside. 

•  Nurses’  pay  structures  and  career  ladders  in  military  facilities  may  vary  greatly 
from  those  in  the  local  community. 

•  Due  to  the  requirement  for  frequent  geographic  relocation  many  military  patients 
and  families  lack  family  support  systems. 

•  Most  patients  seen  in  MTFs  have  full  health  insurance  coverage,  provided  by  the 
MHS. 
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In  addition,  MHS  data  are  not  typically  available  as  large  public  datasets  for  use  in 
studies  comparing  nursing  indicators  to  patient  outcomes.  The  generalizability  of  the 
findings  from  past  nurse  staffing  and  patient  safety  studies  conducted  using  civilian  data 
to  the  MHS  is  questionable. 

It  has  been  noted  that  organizations  measure  what  they  value  (Eccles,  1991 ).  For 
military  and  civilian  health  care  organizations,  it  would  be  more  appropriate  to  state  that 
measurements  are  derived  from  data  that  are  currently  available  (Jennings  &  Staggers, 
1997)  rather  than  what  ought  to  be  measured.  Consequently,  the  reliance  on  individual 
MTF  data  collection  efforts  and  existing  MHS  administrative  databases  to  supply  proxy 
measures  of  clinical  phenomena  yield  gross  estimates  at  best,  as  well  as  distortions  and 
most  often  a  complete  inability  to  address  clinical  issues.  When  hospital  Commanders 
ask  nursing  leaders  to  defend  the  costs  for  the  largest  personnel  pool  in  the  inpatient 
facility,  there  is  often  insufficient  evidence  to  support  a  response. 

Individual  MTF  outcome  data  sets  are  limited  to  descriptive  reports  because  patient 
unusual  occurrences  are  often  uncommon  events.  The  infrequent  occurrence  of  certain 
events  would  require  a  large  number  of  months  or  years  to  accumulate  enough  power 
to  test  a  hypothesis  regarding  the  relationship  of  staffing  to  negative  patient  outcomes. 
From  the  administrative  perspective,  few  facilities  report  data  that  can  be  used  for  inter¬ 
facility  comparison.  This  is  because  many  reported  indicator  values  usually  lack 
definition  specificity. 


Scope  of  the  Study 

This  study  represented  the  fourth  in  a  program  of  research  designed  to  collect  reliable 
and  valid  data  on  nursing  structural  indicators,  nurse-sensitive  patient  outcome 
indicators,  and  nursing  outcomes,  as  well  as  to  explore  the  association  between  nursing 
structural  indicators,  specific  explanatory  variables,  patient  and  nurse  outcomes,  and 
the  context  of  nursing  care  (i.e.,  the  work  environment).  The  first  phase  of  this  research 
program  determined  that  nurse-sensitive  indicator  data  proposed  by  the  American 
Nurses  Association  could  be  successfully  collected  in  one  Army  Medical  Center 
(Hildreth,  Jennings,  Loan,  DePaul,  &  Brosch,  1997;  Jennings,  Loan,  DePaul,  Brosch,  & 
Hildreth,  2001 ).  The  second  in  this  series  of  studies,  the  Army  Nursing  Outcomes 
Database,  demonstrated  that  nursing  indicator  data  and  patient  level  outcomes  could  be 
1 )  standardized  in  terms  of  definitions;  2)  collected  in  two  Army  Medical  Centers;  and  3) 
used  for  decision  making  by  nursing  administration.  Using  California  Nursing  Outcomes 
Classification  (CalNOC)  data  as  a  benchmark,  both  MTFs  were  able  to  compare  their 
staffing,  skill  mix,  and  outcomes  data  to  each  other  and  to  CalNOC  (Brosch  &  Loan, 
2001). 

The  third  study  in  this  program  of  research,  Establishing  a  Military  Nursing  Outcomes 
Database  (MilNOD  III),  successfully  established  that  this  type  of  intense  data  collection 
could  occur  over  a  longer  time  period  (180  days  as  opposed  to  the  previous  study’s  60 
days)  and  could  incorporate  small  and  medium  sized  hospitals  from  all  three  services. 
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Additionally,  definitions  of  key  indicators  were  standardized  across  all  participating 
MTFs  and  the  validity  and  reliability  of  the  data  collected  were  documented.  The 
current  study,  Military  Nursing  Outcomes  Database:  Analysis  and  Expansion  (MilNOD 
IV),  represents  a  shift  in  research  efforts  from  creating  a  high  quality,  reliable  and  valid 
data  collection  mechanism  and  associated  database  to  examining  aspects  of  structure, 
process  and  outcomes  specific  to  nursing. 


Specific  Aims  of  the  Study 

The  study  began  with  the  following  two  specific  aims: 

1 .  Expand  the  number  of  participating  MilNOD  military  treatment  facilities  (MTF)  from 
seven  to  thirteen.  These  would  include  the  following  MTFs  representing  all  three 
branches  of  the  military  (Army,  Navy  and  Air  Force)-*  indicates  new  sites  to  be 
added  to  MilNOD  IV. 

a.  Army  MTFs  -  *Bassett  Army  Community  Hospital,  *Brooke  Army 
Medical  Center,  DeWitt  Army  Community  Hospital,  Madigan  Army 
Medical  Center,  Walter  Reed  Army  Medical  Center,  Womack  Army 
Medical  Center 

b.  Navy  MTFs  -  *National  Naval  Medical  Center  Bethesda,  Naval  Hospital 
Bremerton,  Naval  Hospital  Oak  Harbor,  *Naval  Medical  Center  San 
Diego 

c.  Air  Force  MTFs  -  *Elmendorf  Air  Force  Base  Hospital,  Malcolm  Grow 
Medical  Center,  *Wilford  Hall  Medical  Center 

These  new  sites  were  added  to  allow  further  testing  to  determine  whether  or 
not  the  MilNOD  could  be  replicated  and  deployed  across  additional  military 
treatment  facilities  and  what  utility  could  be  corporately  realized  from  the 
accumulation  of  this  rich  source  of  indicator  and  outcomes  data  across  the 
system. 

2.  Analyze  the  data  collected  during  MilNOD  III  and  IV.  This  analysis  would 
examine  the  relationships  between  nursing  structural  indicators,  contextual 
features  of  the  work  environment,  explanatory  patient  level  variables,  and 
nurse  and  patient  outcome  indicators. 

At  the  time  the  study  was  funded,  the  research  team  proposed  several 
additional  indicator  variables,  and  needed  to  test  their  collection  and  use, 
therefore  aim  #2  was  purposely  vague. 

As  the  study  progressed,  however,  the  specific  aims  were  further  refined  and  separated 
into  two  major  categories:  1 )  expansion  (one  aim)  and  2)  analysis  (ten  aims)  as  follows. 

Expansion 
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The  specific  aim  regarding  the  expansion  of  the  study  was  modified  from  fourteen  total 
sites  to  thirteen  sites  representing  all  three  branches  of  the  military  (Army,  Navy  and  Air 
Force).  One  particular  site  that  was  targeted  for  inclusion,  the  Air  Force  Academy 
hospital,  was  ultimately  excluded  because  of  IRB  and  patient  population  concerns. 

Analysis 

The  analysis  was  further  refined  to  encompass  a  total  of  ten  specific  aims: 

1 .  For  each  unit  type  (medical,  surgical,  ICU,  stepdown)  and  controlling 
for  hospital  size  (small,  large),  do  structural  variables  affect  outcomes 
at  the  shift  level? 

2.  Explore  the  effects  of  patient  turnover  and  census  on  the  relationship 
between  structural  variables  and  outcomes. 

3.  Over  time  have  MilNOD  participating  facilities  decreased  their  use  of 
restraints  and  adverse  events? 

4.  Controlling  for  unit  type,  are  structural  variables  measured  on  day  of 
observed  restraint  assessment  associated  with  restraint  prevalence? 

5.  Controlling  for  unit  type,  are  structural  variables  associated  with 
pressure  ulcer  prevalence  (hospital-acquired  stage  II  and  greater)? 

6.  Controlling  for  patient  risk  (Braden  score,  BUN,  Creatinine,  Albumin), 
determine  which  units  have  a  greater  incidence  of  hospital-acquired 
pressure  ulcers  stage  II  or  greater. 

7.  What  variables,  if  any,  predict  good  versus  poor  performance  related  to 
pressure  ulcer  prevention?  Good  performance  is  indicated  by  having 
high  risk  and  low  pressure  ulcer  prevalence.  Poor  performance  is 
defined  as  low  risk  and  high  pressure  ulcer  prevalence. 

8.  Which  variables  predict  patient’s  report  of  satisfaction  (overall  and 
various  aspects)? 

9.  Does  staffing  and  staff  category  impact  how  nursing  personnel 
respond  when  surveyed  about  the  work  environment  and  nursing  job 
satisfaction? 

10.  Does  patient  turnover  contribute  to  nurse  dissatisfaction? 
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Furthermore,  the  analysis  was  broadened  to  include  not  only  the  data  from  MilNOD  III 
but  the  data  from  MilNOD  IV  as  well.  The  additional  analytic  aims  and  data  inclusion 
plan  was  approved  by  TSNRP  in  2006. 


Research  Plan 
Framework 

The  MilNOD  project  was  guided  by  Donabedian’s  (1966)  triad  of  structure,  process,  and 
outcome;  the  Quality  Health  Outcomes  Model  that  included  feedback  among  patients, 
systems  of  care,  and  interventions  (Mitchell,  Ferketich,  &  Jennings,  1998);  and  the  work 
of  Aiken  and  colleagues  (1997;  2008)  that  incorporates  the  nursing  practice 
environment  as  a  contextual  variable.  MilNOD  researchers  theorize  that  structural 
factors  independently  and  in  combination  with  contextual  factors  affect  patient  and 
nurse  outcomes. 

Most  of  the  indicators  (Table  2)  and  procedures  used  in  MilNOD  are  patterned  after  the 
ANA  Safety  and  Quality  Initiative  (ANA,  1995;  1996a;  1996b)  and  used  in  the  NDNQI. 
However,  the  framework  for  MilNOD  is  specifically  patterned  after  CALNOC  (Brown, 
Donaldson,  Aydin  &  Carlson,  2001). 

Table  2 

MilNOD  Indicators 


Type 

Indicator 

Structural 

Nursing  Care  Hours35 

Nursing  Staff  Mix35 

Nursing  Staff  Education  &  Experience5 

Contextual 

Nursing  Work  Environment 

Explanatory 

Patient  turnover  (admissions,  discharges,  transfers) 

Patient  acuity 

Outcome 

Patient  Pressure  Ulcer  Prevalence313 

Restraint  Use  Prevalence13 
Patient  Falls'3 

Patient  Satisfaction  with  Careab 

Patient  Satisfaction  with  Planning  for  Needs  After  Discharge13 
Patient  Satisfaction  with  Pain  Management35 
Patient  Satisfaction  with  Education35 
Medication  Administration  Errors0 
Nurse  Nursing  Job  Satisfaction35 

_ Nursing  Needlestick  Injuries0 _ 

aMatches  the  NDNQI  definition.  Congruence  with  corresponding  CalNOC  indicator,  developed  during 
MilNOD  III  study. 
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Design 


Expansion 

For  the  expansion,  the  research  team  replicated  the  data  collection  methods  from 
MilNOD  III  and  applied  the  existing  methodology,  to  include  data  reliability  and  validity 
assessments,  to  the  seven  new  MilNOD  IV  sites. 

Analysis 

The  data  used  for  the  analysis  section  of  this  study  consists  of  prospectively  collected 
data  to  include  nurse  staffing,  education,  experience,  and  work  environment  data; 
nursing  job  satisfaction;  and  patient  satisfaction  as  well  as  retrospectively  collected 
adverse  event  reports.  The  data  sources  are  listed  in  Table  3. 

Table  3 

Data  Sources 


Data  Source 

Empirical  Indicator 

Variables 

Daily  staffing 
worksheets 

Institution-specific 

Nursing  staff  hours 

Nursing  skill  mix 

Patient  turnover 

Patient  acuity  (daily) 

Patient  census 

Prevalence 

Prevalence  documentation 

Pressure  ulcer  prevalence 

survey 

form 

Restraint  use  prevalence 

Institutional 

Incident  reports, 

Medication  administration  error 

incident  reports 

institution-specific 

Nursing  staff  needlestick  injury 
Patient  falls 

Patient  survey 

Patient  Satisfaction  with 
Nursing  Care  Questionnaire 

Patient  satisfaction  with: 

Nursing  care 

Planning  for  needs  after 
discharge 

Pain  management 

Education 

Nurse  survey 

Single  item  measure 
Education  and  Experience 
Survey 

Practice  Environment  Scale 
of  the  Nursing  Work  Index 

Nursing  job  satisfaction 

Nursing  staff  education  and 
experience 

Nursing  work  environment 
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Settings 

Data  collection  occurred  at  6  small  (<50  beds)  and  7  large  (>  50  beds)  military  hospitals 
located  throughout  the  United  States.  There  was  one  medium  sized  hospital  (51-99 
beds)  but  its  characteristics  mirrored  those  of  the  larger  facilities  (i.e.,  designated 
medical  center,  teaching  hospital)  so  it  was  subsumed  into  the  large  hospital  category. 
Types  of  units  included  in  the  study  were  medical,  surgical,  mixed  medical/surgical, 
stepdown  and  critical  care  units. 


Units  of  Analysis 

A  variety  of  units  of  measurement  and  analysis  were  used  in  this  study.  These  include: 
Shift-Level 

Nursing  care  hours  for  each  skill  level  (RN,  LNP,  Unlicensed  personnel)  and  category 
(military,  civilian,  contract,  reservist);  nursing  skill  mix;  nursing  category  mix;  patient 
admissions,  discharges,  and  transfers  data  were  collected  at  the  shift-level  by  unit. 
Patient  acuity  and  census  were  collected  at  least  daily.  The  dataset  is  comprised  of 
227,253  shifts  of  staffing,  census,  and  acuity  data. 

Patient  falls  (N=949)  and  nurse  medication  administration  errors  (N=1 ,395)  were 
obtained  from  institutional  adverse  occurrence  reports  and  nurse  needlestick  injuries 
(N=80)  were  obtained  from  occupational  health  or  risk  management  reports.  These 
incidents  were  then  assigned  to  the  unit,  date,  and  shift  of  occurrence. 

Patient-Level 

Pressure  ulcer,  restraint  use  and  patient  satisfaction  with  nursing  care  indicators  were 
assessed  at  the  individual  patient  level.  Patient  satisfaction  data  from  1,721  patients 
are  included  in  the  sample.  In  addition,  approximately  1 ,684  patients  took  part  in 
prevalence  surveys  for  pressure  ulcers  and  restraint  use. 

Nurse-Level 

Nurse  job  satisfaction,  nursing  staff  education  and  experience,  and  the  nursing  work 
environment  were  assessed  at  the  individual  nurse  level.  This  sample  includes  1,042 
RNs  and  544  LPNs  and  unlicensed  assistive  personnel  who  worked  in  medical  surgical, 
critical  care,  or  step-down  units  in  the  thirteen  participating  MilNOD  hospitals. 

Variables  and  Measures. 


Structural  Indicators 

Nursing  care  hours.  Nursing  care  hours  (NCH)  were  defined  as  the  productive  hours 
worked  by  the  inpatient  nursing  staff  who  have  direct  patient  care 
responsibilities/assignments  on  a  defined  unit  and  were  included  in  the  workload 
prediction  system  based  on  patient  volume,  patient  acuity  and/or  nursing  workload 
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(CalNOC,  2001 ).  Direct  patient  care  assignments  were  defined  in  terms  of  those  RNs, 
LPNs  or  unlicensed  assistive  personnel  who  provided  direct  care  for  at  least  50%  of 
their  shift.  When  making  the  decision  about  whether  a  staff  member  should  be  counted 
as  a  direct  patient  care  provider  was  difficult,  coders  were  to  ask  nursing  managers 
“would  the  nurse  be  replaced  if  he  or  she  called  in  sick?”  If  the  answer  was  “yes”  their 
hours  were  included.  Other  paid  hours  for  any  indirect  care  and/or  non-productive  time 
(sick  time,  vacation,  and  education  leave),  committee  time,  or  military  requirements 
(unless  the  time  is  a  very  short  period  of  time  away  from  the  unit  and  those  hours  were 
not  replaced  with  another  direct  patient  care  giver’s  hours)  were  NOT  included.  Nursing 
care  hours  were  collected  in  several  categories: 

•  RN  Care  Hours 

•  LPN  Care  Hours 

•  Other  Care  Hours  (unlicensed  providers) 

•  Total  Nursing  Care  Hours  -  Calculated  within  the  MilNOD  database  from  the 
above  indicators.  Defined  as  the  total  number  of  productive  hours  worked  by 
all  nursing  staff  with  direct  patient  care  responsibilities  (RN,  LPN,  aides,  other 
direct  care  providers  included  in  the  staffing  matrix).  These  hours  were 
documented  each  shift. 

Nursing  Skill  Mix.  Nursing  skill  mix  was  defined  as  the  relative  proportion  of  total 
nursing  care  hours  delivered  by  unique  categories  of  nursing  providers. 

•  RN  Skill  Mix  -  The  proportion  of  RN  nursing  care  hours  compared  to  total 
nursing  care  hours. 

•  LPN  Skill  Mix  -  The  proportion  of  LPN  nursing  care  hours  compared  to  total 
nursing  care  hours. 

•  Other  Skill  Mix  (unlicensed  provider  mix)  -  The  proportion  of  all  unlicensed 
nursing  care  hours  compared  to  total  nursing  care  hours.  Nursing  assistants 
were  the  most  predominant  care  providers  in  this  category.  Other  providers 
such  as  telemetry  technicians  on  a  cardiac  step-down  unit  and  Air  Force  and 
Navy  corpsmen  were  annotated  in  this  category. 

Nursing  category  mix.  Nursing  personnel  work  hours  were  further  divided  by  category 
of  provider,  i.e. ,  active  military,  GS  civilian,  contractor,  and  military  reservist.  These 
categories  were  calculated  as  proportions  similar  to  the  skill  mix  as  indicated  above. 

Nursing  Education  and  Experience.  Nursing  personnel  included  RNs,  LPNs  and 
unlicensed  personnel.  This  information  provided  an  additional  dimension  in  the 
interpretation  of  nurse  staffing  information  and  was  collected  from  individual  nurses 
using  an  adaptation  of  the  CalNOC  Education  and  Experience  Questionnaire. 
Components  included: 

•  Demographic  Information 

•  Highest  education  level  Highest  level  of  nursing  education 

•  Number  of  years  of  nursing  experience 

•  Number  of  years  of  experience  in  the  current  hospital 
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•  Number  of  years  of  experience  taking  care  of  the  types  of  patient 
encountered  in  currently  assigned  unit. 

•  Certifications 

Patient  Outcome  Indicators 

Pressure  Ulcer  Prevalence.  This  was  defined  as  the  proportion  of  all  patients 
examined  during  a  one-day  prevalence  survey  with  stage  I,  II,  III,  IV  pressure  ulcers.  All 
inpatients  admitted  prior  to  midnight  of  the  prevalence  survey  day  were  included.  This 
included  those  patients  admitted  with  pressure  ulcers.  Prevalence  is  expressed  as  a 
percentage,  in  relation  to  the  total  number  of  patients  surveyed  (#  patients  with  ulcers/  # 
patients  in  study).  Note  that  prevalence  is  NOT  the  number  of  ulcers  discovered  (some 
patients  have  multiple  ulcers)  (CalNOC,  2001 ).  A  copy  of  the  tool  used  to  gather  the 
survey  data  is  included  in  this  report.  Hospital  acquired  pressure  ulcers  prevalence  is 
defined  as  the  proportion  of  all  patients  examined  with  a  Stage  II  or  greater  pressure 
ulcer  that  was  not  documented  on  admission. 

Restraint  Use  Prevalence.  This  was  defined  as  the  proportion  of  all  patients 
observed  (on  the  day  of  the  pressure  ulcer  prevalence  study)  who  had  one  or  more 
restraints  in  place.  Prevalence  is  expressed  as  a  percent,  in  relation  to  the  total  number 
of  patients  surveyed  (#  patients  with  restraints/#  patients  in  study).  The  definition  of  a 
restraint  used  was  “any  method  of  physically  restricting  a  person’s  freedom  of 
movement,  physical  activity,  or  normal  access  to  his  or  her  body  either  part  of  an 
approved  protocol,  or  as  indicated  by  individual  order”  (CalNOC,  2001).  This  included  4 
bed  rails  in  the  up  position.  A  copy  of  the  tool  used  to  gather  the  survey  data  is  included 
with  this  report. 

Patient  Falls.  A  patient  fall  is  defined  as  “a  patient’s  unplanned  descent  to  the 
hospital  floor”  (CalNOC,  2001).  Falls  data,  extracted  from  MTF  unusual  occurrence 
reports,  included  the  unit  on  which  the  fall  occurred,  the  time  of  day  of  the  fall,  the 
presence  of  and  level  of  injury,  circumstances  (observed,  assisted,  restrained  at  the 
time  of  the  fall),  type  of  fall  (accidental,  unanticipated  physiologic,  anticipated 
physiologic  fall  or  unknown;  Morse,  1991)  and  presence  or  absence  of  falls  prevention 
protocol  initiation.  A  patient  fall  with  injury  was  treated  as  a  separate  outcome  variable 
and  was  defined  as  a  fall  with  ANY  injury  to  the  patient  as  documented  on  the  hospital 
unusual  occurrence  report. 

Medication  Administration  Error.  A  medication  administration  error  is  defined  as 
“a  deviation  from  the  physician’s  medication  order  as  written  on  the  patient’s  chart” 

(Allan  &  Barker,  1990,  p.555)  committed  by  a  nurse.  Medication  error  data  were 
extracted  from  institutional  incident  reports.  Any  “near  miss”  errors,  intercepted  and 
corrected  before  reaching  the  patient,  were  not  counted  as  an  actual  medication 
administration  error.  Data  that  were  collected  include  the  unit  where  the  error  occurred, 
date,  time,  type  of  medication  error,  and  presence  and  level  of  injury. 
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Patient  Satisfaction  with  Nursing  Care.  The  Patient  Satisfaction  with  Nursing 
Care  Quality  (PSNQ;  Jacox,  Bausell,  &  Merenholtz,  1997)  survey  was  used  to  measure 
patient  satisfaction  with  aspects  of  hospital  care.  The  CalNOC-developed  definitions  of 
four  specific  aspects  of  patient  satisfaction  derived  from  the  ANA  Nursing  Quality 
Indicators  were  used  in  this  study.  However,  CalNOC  used  Yes/No  responses  which 
we  deemed  not  appropriate  to  capture  sufficient  variability.  To  remain  consistent  with 
CalNOC  and  NDNQI  patient  satisfaction  with  nursing  care  measures,  items  identical  to 
those  included  on  the  CalNOC  patient  satisfaction  survey  were  pulled  from  the  PSNQ 
instrument.  In  addition,  we  also  report  the  orginal  PSNQ  subscales. 

•  Patient  Satisfaction  with  Pain  Management:  A  measure  of  patient  perception  of 
the  hospital  experience  related  to  satisfaction  with  pain  management. 

•  Patient  Satisfaction  with  Patient  Education:  A  measure  of  patient  perception  of 
the  hospital  experience  related  to  satisfaction  with  patient  education. 

•  Patient  Satisfaction  with  Planning  for  Needs  after  Discharge:  A  measure  of 
patient  perception  of  the  hospital  experience  related  to  satisfaction  with  planning 
for  needs  after  discharge. 

•  Patient  Satisfaction  with  Overall  Care:  A  measure  of  patient  perception  of  the 
hospital  experience  related  to  satisfaction  with  overall  care. 

Nursing  Staff  Outcome  Indicators 

Nursing  Job  Satisfaction.  Nursing  job  satisfaction  is  defined  as  the  degree  to 
which  a  nurse  rates  his  or  her  global  contentment  with  her  job.  Job  satisfaction  was 
measured  in  this  study  by  a  single  item-measure  of  overall  job  satisfaction  which  was 
found  to  be  highly  correlated  with  respondent’s  global  satisfaction  scores  in  other 
studies  (Patrician,  2004)  The  item  reads:  Overall,  how  satisfied  are  you  with  your 
current  job?  The  response  choices  are:  5  =  Very  Satisfied,  4  =  Somewhat  Satisfied, 

3  =  Neutral,  2  =  Somewhat  Dissatisfied,  and  1  =  Very  Dissatisfied. 

Nursing  Staff  Needlestick  Injury.  A  nursing  staff  needlestick  injury  is  defined  as 
“a  puncture  with  a  needle  or  sharp  instrument  that  is  contaminated  with  blood”  (Clarke, 
Sloan,  &  Aiken,  2002).  Needlestick  injuries  were  obtained  from  the  occupational  health 
clinic  or  its  equivalent  of  the  participating  MTF.  Data  on  needlestick  injuries  included 
time,  date,  unit,  personnel,  type,  device,  and  whether  or  not  the  device  was 
contaminated. 

Explanatory  Variables 

Patient  Turnover.  Patient  turnover  is  defined  as  the  number  of  admissions, 
discharges  and  transfers  (ADT)  for  the  past  shift  divided  by  the  unit  patient  census.  It  is 
also  referred  to  as  the  ADT  Index.  Census  is  a  static  number-it  does  not  reflect  the 
considerable  work  generated  by  admitting  patients  to  the  unit,  discharging  patients  from 
the  unit,  or  transferring  to  or  from  another  unit  (Fralic,  2000).  The  ADT  Index  is  a 
pragmatic  and  easily  understood  way  to  reflect  the  stress  and  strain  on  nursing  staff  that 
is  not  always  reflected  in  the  census.  The  census  by  itself  is  far  less  informative  than 
the  census  viewed  in  combination  with  the  ADT  index.  A  high  census  with  a  low  ADT 
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Index  reflects  a  more  stable  workload  than  a  high  census  in  combination  with  a  high 
ADT  Index.  Conversely,  a  low  census  with  a  high  ADT  Index,  especially  when  the  high 
index  is  sustained  reflecting  a  high  turnover  of  patients,  can  reflect  situations  in  which 
staff  are  exposed  to  unrelenting  stress,  a  condition  that  may  set  the  stage  for 
compromising  the  quality  of  care.  Every  shift,  the  staff  were  expected  to  enter  these 
data  into  the  database. 

Patient  Acuity.  Patient  acuity  is  defined  as  the  severity  of  a  patient’s  illness  and 
reflects  nursing  care  requirements  of  patients.  The  Workload  Management  System  for 
Nursing  (WMSN)  is  the  acuity  system  that  has  been  used  by  all  three  services  since  the 
1980s  (WMSN,  n.d.)  Developed  from  time  and  motion  studies,  it  relies  on  a  very 
detailed  checklist  of  nursing  tasks.  Once  a  checklist  is  completed  and  points  are  totaled, 
a  number  is  assigned  from  I  to  VI  to  indicate  the  nursing  care  requirements  for  a 
particular  patient.  The  unit’s  total  patient  acuity  has  been  converted  into  required 
nursing  care  hours  and  has  been  used  as  a  staffing  projection  system  for  many  years. 
The  staffing  predictive  capability  of  the  WMSN  has  been  questioned,  and  therefore 
required  nursing  care  hours  were  not  used  in  this  study.  However,  the  individual  acuity 
measure,  specifically  the  average  unit  acuity  was  thought  to  be  useful  for  this  study. 
Nurses  were  asked  to  enter  into  their  unit’s  database  the  number  of  patients  within  each 
acuity  category  and  an  average  acuity  measure  was  tabulated.  This  was  generally 
done  once  a  day. 

Contextual  variable:  Nursing  work  environment.  The  nursing  work  environment  was 
defined  as  “conditions  which  facilitate  or  detract  from  the  ability  of  nurses  to  carry  out 
their  work”  (Lake,  2002).  Work  environment  was  measured  with  t  he  Practice 
Environment  Scale  of  the  Nursing  Work  Index  (PES-NWI;  Lake,  2002).  On  the  PES- 
NWI,  nurses  indicated  the  extent  to  which  certain  work  environment  attributes  were 
present  in  their  current  job.  Items  comprising  the  PES-NWI  scales  are  ranked 
qualitatively  with  four  category  responses  ranging  from  one  (strongly  disagree)  to  four 
(strongly  agree)  with  a  midpoint  of  2.5.  From  the  PES-NWI,  five  subscales  were 
calculated:  Nurse  Participation  in  Hospital  Affairs;  Nursing  Foundations  for  Quality  of 
Care;  Nurse  Manager  Ability,  Leadership  and  Support;  Staffing  and  Resource 
Adequacy;  and  Collegial  Nurse-Physician  Relations.  A  Professional  Practice 
Composite  Score  was  also  calculated  to  represent  an  overall  measure  of  the  work 
environment.  Reliability  and  validity  of  the  instrument  and  its  subscales  have  been 
published  (Lake,  2002). 


Data  Collection  Methods 


Data  collection  overview 

Participation  in  this  multi-site  study  was  solicited  by  letters  of  invitation  directed  at  Chief 
Nursing  Officers  (CNOs)  at  the  proposed  hospitals.  Following  IRB  approval  and 
hospital  enrollment,  core  MilNOD  team  members  visited  each  site  to  introduce  the  study 
to  major  stakeholders,  including  the  Chief  Nurse,  section  supervisors/division  heads, 
and  unit  level  nurse  managers.  Research  assistants  (RAs)  or  site  coordinators  (SCs) 


24 


Col  (ret)  Patrician  A.  Patrician 


Proposal  No  N03-P07 


were  situated  either  directly  at  the  facility  (for  large  facilities)  or  within  geographic 
clusters  of  hospitals  (for  small  facilities)  to  serve  as  facilitators  for  data  collection.  Each 
SC/RA  received  a  set  of  orientation  materials  in  addition  to  the  Codebook  and  was 
trained  extensively  by  core  team  members.  SC/RAs  then  walked  individual  site  staff 
through  the  process  of  data  collection  and  data  submission  initially  and  whenever  site 
personnel  were  replaced.  A  30-day  run-in  period  to  ensure  data  accuracy  and  validity 
preceded  data  collection  at  each  site.  Feedback  on  accuracy  and  completeness  of  data 
was  provided  on  a  continual  basis. 

Instructions  for  all  survey  processes  were  standardized  with  oversight  provided  by  COL 
(ret)  Bingham  at  Brooke  Army  Medical  Center  and  survey  development  and  distribution 
provided  by  Mr.  Jim  Williams  (at  Walter  Reed  Army  Medical  Center).  Training  materials 
were  made  available  to  each  site  as  needed. 

Data  collection  schedule 

Timelines  were  provided  to  site  coordinators  were  used  to  assist  them  with  organization 
and  coordination  of  study  activities.  Surveys  were  conducted  in  a  designated  quarter. 
Every  attempt  was  made  to  have  all  facilities  conduct  the  survey  as  scheduled  so  that  a 
report  of  findings  from  the  survey  could  include  comparisons  from  other  like-size 
hospitals.  Table  4  highlights  the  data  collection  schedule. 


Table  4 

Data  Collection  Schedule 


Quarter  of  FY 

Months 

Data  Collected 

1st 

October-December 

Nursing  Survey 

2nd 

January-March 

Pressure  Ulcer/Restraint  Prevalence  (a) 

3^ 

April-June 

Patient  Survey 

4^ 

July-September 

Pressure  Ulcer/Restraint  Prevalence  (b) 

Ongoing 

Monthly  transmittal  of  shift  level  data 
approximately  8  weeks  after  the  end  of  a 
quarter;  quarterly  transmission  of  adverse 
event  data 

Structural  Indicators 

Staffing  measures  were  captured  for  each  shift  using  the  traditional  8-hour  shift 
categories  (day  (0700-1459),  evening  (1500-2259),  and  night  (2300-0659).  Every  shift, 
the  nurse  manager,  or  designee,  reported  the  hours  worked  by  each  of  the  following 
provider  types:  RNs,  LPNs,  and  unlicensed  personnel,  which  included  NAs,  corpsmen, 
and  telemetry  technicians. 
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Reflecting  the  unique  configuration  of  military  hospital  staffing,  each  provider  type  was 
further  differentiated  by  the  following  categories:  active  duty  military,  Department  of 
Defense  civilian,  military  reservist,  or  contract/agency.  The  reservist  category  was 
comprised  of  nurses  who  were  activated  in  support  of  Operation  Enduring 
Freedom/Operation  Iraqi  Freedom  and  often  times  replaced  deployed  military  nurses. 
For  those  staff  who  worked  12  hour  shifts,  staff  work  hours  were  split  into  the  two  time 
frames  that  encompassed  the  shift,  typically  with  four  hours  on  one  shift  and  eight  on 
the  other.  One  hospital  in  particular  had  14  different  shift  configurations.  Instruction 
sheets  with  conversions  assisted  data  entry  personnel  in  transforming  the  various  shifts 
into  8-hour  increments. 

The  unit  managers  or  designated  data  entry  personnel  were  instructed  not  to  count 
hours  providers  spent  away  from  the  unit,  i.e.,  time  spent  in  a  class  or  as  borrowed 
manpower  “floated”  to  another  unit  in  the  hospital.  Similarly,  hours  worked  by  nursing 
personnel  on  loan  to  a  specific  unit  or  those  providing  consultation  on  that  unit  such  as 
a  wound  care  nurse,  were  counted  as  present  on  that  unit. 

From  the  data  on  patient  care  hours  worked,  the  researchers  calculated  percentages  for 
provider  type  (skill  mix),  i.e.,  RN,  LPN,  NA  and  provider  category,  i.e.,  civilian,  military, 
reservist,  contract.  Total  nursing  care  hours  per  patient  shift  was  a  sum  total  of  all 
hours  worked  by  all  nursing  providers  for  that  shift  divided  by  number  of  patients  on  the 
shift.  The  remaining  structural  variables,  to  include  nursing  staff  education  and 
experience,  were  collected  via  survey  and  methods  used  are  described  under  “Survey” 
below. 

Explanatory  Variables 

Explanatory  variables  include  patient  census,  average  patient  acuity  (based  upon  a 
standard  acuity  system  used  by  the  military),  and  patient  turnover.  Patient  census  was 
captured  each  shift;  however,  when  any  given  shift  census  was  missing,  the  daily 
census  for  that  day  was  used.  Patient  acuity  data  was  captured  on  the  day  shift. 
Because  the  patient  acuity  system  was  designed  to  be  a  prospective  acuity  system,  all 
three  shifts  on  a  given  day  were  assigned  the  same  acuity  values  captured  for  that  day. 
Admissions,  discharges  and  transfers  were  captured  each  shift  and  used  to  calculate 
the  ADT  Index  as  previously  described. 

Outcome  Indicators 

A  separate  database  was  created  and  maintained  for  adverse  events.  Each  month 
Performance  Improvement  (PI)  data  (institutional  incident  reports),  for  the  monthly 
period  starting  three  months  prior  to  the  month  of  collection,  were  reviewed  and  data 
were  extracted.  A  three  month  lag  time  was  used  to  ensure  that  all  incident  reports 
traversed  the  system  and  were  available  in  the  PI  office.  Information  that  was  collected 
by  the  “events”  database  included  the  date  and  time  of  the  incident,  whether  or  not  the 
patient  was  harmed  and  the  level  of  harm,  types  of  falls  (anticipated,  unanticipated, 
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accidental),  types  of  medication  administration  errors,  and  whether  a  nurse  sustained  a 
needlestick  injury  with  a  contaminated  needle. 

In  order  to  analyze  the  associations  between  staffing  and  the  occurrence  of  adverse 
events,  the  “events”  dataset  was  merged  with  the  shift  level  staffing  and  census 
database.  Medication  administration  errors  (ME),  patient  falls,  falls  with  injury,  and 
needlestick  injuries  (NS)  were  coded  as  either  0  or  1 ,  indicating  the  absence  or 
presence,  respectively,  of  the  particular  event  each  shift.  In  addition,  a  shift  composite 
outcome  was  created,  defined  as  the  presence  of  any  adverse  occurrence  (AO)  out  of 
those  events  listed  above.  Inter-rater  reliability  scenarios  were  developed  to  assess 
validity  and  reliability  of  falls  and  medication  error  reporting. 

Prevalence  Studies 

Pressure  ulcer  and  restraint  use  prevalence  studies  were  conducted  at  each  of  the 
participating  sites  by  onsite  staff  and  members  of  the  study  team  using  direct  patient 
observation  and  medical  record  review.  MilNOD  study  team  members  and  local  wound 
nursing  experts  taught  on-site  nurses  how  to  grade  pressure  ulcers  and  evaluate 
restraint  use.  On  the  day  of  each  prevalence  study,  all  participating  adult  acute  care 
inpatients  admitted  to  the  MTF  prior  to  midnight  the  night  before  received  a  full  body 
skin  assessment  and  evaluation  of  restraint  use.  Inter-rater  reliability  was  conducted 
with  pressure  ulcer  and  restraint  use  prevalence  surveys.  A  member  of  the  MilNOD 
team  was  present  to  assist  with  training  and  conducting  the  PU  survey  initially  at  each 
site  and  for  subsequent  surveys  upon  request  (usually  Dr.  McCarthy  and/or  LTC 
Armstrong).  Retrospective  data  related  to  pressure  ulcers  and  restraint  use  were 
collected  from  the  inpatient  records  of  participating  patients.  During  the  prevalence 
study,  all  skin  and  restraint  evaluations  were  performed  by  at  least  two  trained  nurses  to 
further  ensure  agreement  and  inter-rater  reliability.  Table  5  details  the  pressure  ulcer 
and  restraint  use  survey  dates  and  level  of  participation  by  MTF.  Each  patient 
assessed  for  pressure  ulcers  was  also  assessed  for  restraint  use. 

Table  5 

Pressure  Ulcer  and  Restraint  Use  Prevalence  Study  Assessment  Rates  by  Facility  and 
Year 


Facility 

Date 

#  Eligible 
patients 

#  Patients 
Assessed 

Proportion 

Assessed 

101 

19  Aug  2003 

165 

125 

76% 

9  Jun  2004 

140 

115 

82% 

Jan  2005 

151* 

113 

75%* 

Jan  2006 

130* 

100 

77%* 

102 

25  Sep  2003 

68 

67 

99% 

27  Jul  2004 

89 

83 

93% 

23  Feb  2005 

102 

91 

89% 

Oct  2005 

76* 

65 

86%* 
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Mar  2006 

80* 

79 

99%* 

103 

27  Apr  2005 

156 

136 

87% 

7  Jun  2006 

147* 

147 

100%* 

104 

17  Nov  2005 

104 

104 

100% 

105 

Apr  2006 

77* 

77 

100%* 

106 

Sep  2005 

— 

75 

— 

Mar  2006 

87* 

69 

79%* 

501 

1 1  Dec  2003 

41 

39 

95% 

7  Jun  2004 

39 

30 

77% 

Apr  2005 

42* 

42 

100%* 

Feb  2006 

39* 

35 

90%* 

502 

22  Sep  2003 

26 

24 

92% 

7  Jun  2004 

17 

12 

71% 

Jan  2005 

12* 

12 

100%* 

Feb  2006 

12* 

8 

67%* 

503 

Mar  2006 

5* 

4 

80%* 

901 

17  Dec  2003 

16 

13 

81% 

Aug  2004 

1 

1 

100% 

Jan  2005 

4 

3 

75% 

903 

4  Jun  2004 

7 

4 

57% 

904 

4  Jun  2004 

11* 

11 

100%* 

TOTALS 

1,844 

1,684 

Ave  91% 

*  Estimated  census  from  average  daily  census  of  the  month  the  PU  Prevalence 
study  was  conducted.  There  were  no  census  data  for  Facility  106  in  September  2005. 


Nursing  Survey 

All  MTFs  administered  the  Nursing  Personnel  Education,  Experience,  and 
Certification  Survey,  the  Revised  Nursing  Work  Index  Survey,  and  a  single  item  nurse 
satisfaction  measure  to  the  nursing  staff  on  participating  units  at  approximately  twelve 
month  intervals  using  a  modified  Dillman  (2007)  method.  The  Dillman  method 
recommends  an  advance  letter  to  let  potential  participants  know  that  a  survey  will  soon 
be  mailed  to  them.  This  step  was  modified  slightly  by  announcing  the  survey  to  nursing 
staff  through  the  MTF  email  system.  Additionally,  the  nurse  surveys  were  not  mailed. 
Instead  surveys  with  an  attached  stamped  return  envelope  were  placed  in  each  nurse’s 
mailbox  on  their  unit.  Nurses  were  asked  to  fill  out  the  survey  during  their  duty  time  and 
instructed  to  mail  their  completed  survey  in  the  return-addressed  stamped  envelope  to 
the  study  PO  Box  in  Laurel,  MD.  Also  as  advocated  by  Dillman  (2007),  a  post  card  and 
email  message  were  sent  to  all  nurses  approximately  two  weeks  after  the  first  survey 
distribution.  This  follow-up  procedure  served  to  thank  people  who  participated  and 
encouraged  those  who  had  not  yet  returned  their  survey  to  do  so.  Finally,  two  weeks 
after  the  post  card  was  distributed,  a  second  distribution  of  surveys  was  made  to  all 
non-responders  from  the  first  distribution  and  the  follow-up  postcard.  Return  envelopes 
on  all  surveys  were  coded  to  allow  survey  central  research  team  members  to  determine 
which  nurses  required  a  second  survey. 
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Table  6 

Nurse  Survey  Response  Rates  by  Facility  and  Year 


Facility 

Year 

RN 

Surveys 

LPN/NA 

Surveys 

Sent 

Return 

Response 

Rate 

Sent 

Return 

Response 

Rate 

101 

2003 

212 

79 

37.3% 

200 

50 

25.0% 

2004 

219 

77 

35.2% 

186 

42 

22.6% 

2005/6 

219 

68 

31.1% 

166 

30 

18.1% 

102 

2003 

158 

66 

41.8% 

150 

39 

26.0% 

2004 

165 

61 

37.0% 

127 

39 

30.7% 

2005/6 

139 

66 

47.5% 

92 

44 

47.8% 

103 

2004 

218 

98 

45.0% 

117 

73 

62.4% 

2005/6 

270 

105 

38.8% 

163 

46 

28.2% 

104 

2005/6 

244 

100 

40.9% 

98 

20 

20.4% 

105 

2005/6 

124 

32 

25.8% 

73 

8 

10.9% 

106 

2005/6 

144 

66 

45.8% 

126 

42 

33.3% 

501 

2003 

150 

38 

25.3% 

100 

15 

15.0% 

2004 

55 

28 

50.9% 

16 

7 

43.8% 

2005/6 

59 

26 

44.1% 

38 

18 

47.4% 

502 

2003 

60 

37 

61.7% 

75 

12 

16.0% 

2004 

44 

21 

47.7% 

48 

9 

18.8% 

2005/6 

27 

11 

40.7% 

27 

5 

18.5% 

503 

2005/6 

23 

20 

87.0% 

33 

21 

63.6% 

901 

2003 

30 

8 

26.7% 

35 

7 

20.0% 

2004 

17 

8 

47.1% 

18 

6 

33.3% 

2005/6 

17 

4 

23.5% 

14 

5 

35.7% 

902 

2005/6 

16 

10 

62.5% 

15 

3 

20.0% 

903 

2004 

16 

10 

62.5% 

4 

1 

25.0% 

2005/6 

10 

3 

30.0% 

8 

2 

25.0% 

TOTALS 

2,636 

1042 

39.5% 

1,929 

544 

28.2% 

Patient  satisfaction  survey 

Patient  satisfaction  surveys  were  conducted  using  the  process  described  above 
with  a  few  modifications.  All  patients  discharged  to  home  from  participating  units  at  a 
participating  MTF  after  1  May  2004  were  mailed  a  patient  satisfaction  survey  at 
specifically  designated  times,  according  to  the  predetermined  survey  schedule.  On-site 
coordinators  sent  the  names  of  discharged  patients  and  their  mailing  addresses  to  the 
WRAMC  research  team  within  one  week  of  discharge.  At  two  facilities  it  was  required 
that  the  RA  travel  to  the  MTF  and  prepare  the  survey  packets  on  site  because  the  IRB 
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Committee  at  those  facilities  did  not  want  patient  names  to  be  released  from  the  MTF. 
For  all  other  facilities  the  survey  central  office  mailed  surveys  to  patients  two  weeks 
after  their  discharge.  Patients  received  a  survey  with  a  return-addressed  stamped 
envelope.  They  were  instructed  to  return  the  survey  via  U.S.  Postal  Service  in  the 
envelope  provided.  Post  cards  and  a  second  survey  were  mailed  to  these  patients  as 
previously  described  for  the  nursing  surveys.  The  number  of  patients  surveyed  and  the 
response  rate  is  provided  in  Table  7. 

Table  7 

Patient  Satisfaction  Survey  Response  Rates  by  Facility  and  Year 


Facility 

Date 

#  Patients 
Surveyed 

#  Patients 
Responded 

Response 

Rate 

101 

2004 

280 

143 

51.0% 

2005 

349 

144 

41.2% 

2006 

183 

84 

45.9% 

102 

2004 

150 

108 

72.0% 

2005 

200 

147 

73.5% 

2006 

195 

133 

68.2% 

103 

2005 

200 

126 

63.0% 

2006 

180 

122 

67. 8& 

105 

2006 

190 

130 

68.4% 

106 

2006 

183 

117 

63.9% 

501 

2005 

181 

94 

51.9% 

2006 

163 

95 

58.3% 

502 

2004 

150 

51 

34.0% 

2005 

37 

18 

48.6% 

2006 

38 

17 

44.7% 

503 

2006 

39 

15 

38.5% 

901 

2004 

115 

43 

37.4% 

2005 

40 

25 

62.5% 

2006 

48 

24 

50.0% 

902 

2006 

46 

18 

39.1% 

903 

2005 

50 

25 

50.0% 

2006 

36 

16 

44.4% 

904 

2006 

53 

26 

49.1% 

TOTALS 

3,106 

1,721 

55.4% 

The  research  team  members  responsible  for  survey  administration  used 
Teleform®  software  to  prepare  the  surveys,  scan  completed  surveys,  and  enter  data 
that  were  exported  to  a  designated  file  into  SPSS.  The  survey  forms  used  a 
combination  of  numeric  constrained  print  fields  and  choice  fields  and  participants 
darkened  selected  responses.  The  form  design  defined  how  the  data  were  validated 
and  stored  in  the  database,  including  variable  types  and  coding  for  single  or  multiple 
responses.  When  forms  were  scanned  into  the  software,  the  Teleform®  Reader 
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automatically  evaluated  the  record  and  either  held  it  for  verification  or  interpreted  the 
characters,  darkened  bubbles,  and  other  markings.  This  verification  procedure  required 
meticulous  attention  to  detail,  including  matching  data  received  with  transmission 
information  from  the  MTFs  to  ensure  no  data  were  lost  in  the  mail.  The  verified  Excel 
data  files  were  then  checked  for  errors  and  imported  into  the  MilNOD  data  bank. 

Data  Preparation 

Data  Quality  Assessments.  Preparation  for  data  analysis,  including  data  cleaning, 
identification  of  outliers,  and  data  integrity  checks,  was  originally  accomplished  at 
MAMC.  Throughout  the  study,  the  research  staff  at  MAMC  continually  cleaned  the  data 
and  assessed  for  out  of  range  elements.  Apparent  errors  and  out  of  range  entries  were 
verified  by  phone  call  with  on  site  Research  Assistants  and/or  directly  with  unit  nurse 
managers.  Quarterly  reports  were  another  opportunity  to  conduct  data  quality 
assessments  as  nurse  leaders  at  the  sites  were  able  to  visualize  their  data  and  unusual 
data  elements  were  brought  to  the  attention  of  the  MAMC  research  staff.  In  an  effort  to 
continually  clean  the  database,  staff  were  allowed  to  change  the  data  entered  in  the 
database  if  it  could  be  verified.  For  example,  if  a  unit  appeared  to  have  too  many  falls, 
the  RA  could  go  back  to  source  data  to  track  down  the  error  and  it  was  reconciled  in  the 
database.  Any  requested  changes  o  the  database  that  were  not  based  on  evidence 
were  not  made. 

The  quarterly  reports  also  contained  missing  data  reports  to  alert  the  nurse  leaders  to 
the  absence  of  data,  and  thresholds  were  set  in  reporting,  such  that  if  greater  that  70% 
of  the  data  were  missing  on  a  variable,  a  report  would  not  be  generated  for  that  data 
element. 

Analytic  Database  Preparation.  Merging  of  the  data  for  analysis  was  conducted  at 
MAMC  and  also  at  the  University  of  Alabama.  For  the  first  five  aims  of  the  analysis,  the 
shift  level  dataset  was  constructed  at  MAMC,  and  had  undergone  further  cleaning  and 
verifying  between  the  analyst  and  MAMC  database  experts.  Because  we  needed  to  tie 
adverse  events  with  staffing,  the  adverse  event  data  had  to  be  merged  with  the  shifts  in 
which  they  occurred. 


Data  Analysis 

In  order  to  analyze  the  associations  between  staffing  and  the  occurrence  of  adverse 
events,  the  "events"  database  was  merged  with  the  shift  level  staffing  and  census 
database.  Medication  administration  errors  (ME),  patient  falls,  falls  with  injury,  ad 
needlestick  injuries  (NS)  were  coded  as  either  0  or  1 ,  indicating  the  absence  or 
presence,  respectively,  of  the  particular  adverse  event  each  shift.  In  addition,  a  shift 
composite  outcome  was  created,  defined  as  the  presence  of  any  adverse  occurrence 
(AO)  out  of  those  events  explained  above. 
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Analysis  Aim  #1 :  For  each  unit  type  (medical,  surgical,  ICU,  stepdown)  and 
controlling  for  hospital  size  (small,  large),  do  structural  variables  affect 
outcomes  at  the  shift  level? 

At  the  outset  of  analysis,  the  entire  data  set  was  evaluated  for  extreme  and  missing 
data.  Extreme  data  elements,  assumed  to  be  data  entry  errors,  were  recoded  as 
missing.  Last  value  carried  forward  was  used  to  impute  missing  census  (7%  of  shifts), 
staffing  values  (2%  of  shifts),  and  patient  acuity  (35%  of  days)  information.  This  method 
was  chosen  on  the  basis  of  observations  that  census  and  patient  acuity  values  were 
positively  autocorrelated.  Shifts  missing  all  outcome  measures  were  excluded  from  all 
analysis,  and  those  with  partial  outcome  information  were  excluded  only  from  analysis 
where  the  specific  outcome  was  missing. 

Because  multiple  outcomes  per  shift  were  extremely  rare  (e.g.,  0.08%  of  shifts  had  2 
medication  errors  and  less  than  half  of  that  had  3),  all  outcomes  were  recoded  into 
dichotomous  variables,  0  or  1,  indicating  the  absence  or  presence,  respectively,  of  an 
adverse  event  on  a  shift.  The  probability  of  each  adverse  event  was  modeled  by  using 
hierarchical  logistic  regression  because  the  outcome  variables  were  dichotomous.  This 
modeling  framework  facilitates  the  analysis  of  multi-level  (clustered)  data  by 
decomposing  the  overall  variation  in  outcome  attributable  to  each  level  while 
acknowledging  the  intra-cluster  correlations.  In  this  analysis,  we  used  three  data  levels: 
shift  (lowest),  days,  and  nursing  units  (highest).  Each  data  level  has  its  own  error  term, 
so  that  the  model  could  separate  the  three  sources  of  variation  (i.e.,  at  the  shift,  day, 
and  unit  levels). 

We  explored  the  relationship  of  outcomes  to  shift  time-of-day  using  indicators  for  the 
three  day  periods  (morning,  evening,  night)  and  to  day  of  the  week  by  incorporating 
indicator  variables  for  each  day  and  for  grouped  days  based  on  the  resulting  similarities 
in  their  estimated  coefficients.  Yearly  effects  were  measured  with  indicator  variables  for 
each  study  year,  2003  to  2006. 

It  is  well  known  that  smaller  hospitals  have  many  differences  in  patient  care 
characteristics  compared  to  larger  ones  (e.g.,  less  specialization,  differences  in 
organizational  and  structural  factors,  differences  in  working  conditions,  differences  in 
staffing  and  skill  mix,  etc.).  Therefore,  we  adjusted  for  small  and  large  hospital  size, 
defined  as  50  beds  or  less  and  100  beds  or  more,  respectively.  There  was  one  medium 
sized  hospital,  defined  as  51  to  99  beds.  Since  its  characteristics  mirrored  the  larger 
hospitals  (teaching  hospital  and  designated  medical  center),  it  was  subsumed  into  the 
large  hospital  category.  Since  there  were  no  additional  unit  or  hospital  level  covariates, 
and  to  avoid  a  more  complex  model  with  four  levels,  hospital  size  was  included  as  a  unit 
level  variable. 

The  models  were  estimated  under  a  Bayesian  framework  that  assigns  non-informative 
prior  probability  distributions  to  all  unknown  parameters.  Posterior  distributions  of  the 
model  parameters  (conditional  on  the  data)  were  derived  by  using  Markov  Chain  Monte 
Carlo  methodology.  For  each  outcome,  we  used  a  single  Gibbs  sampler  string 
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implemented  with  WinBUGS  software  (Spiegelhalter,  Thomas,  Best,  &  Lunn,  2003), 
with  a  burn-in  of  500  iterations  and  with  a  further  4500  iteration  used  for  inference. 
Starting  values  for  parameters  were  calculated  using  standard  logistic  regression 
models.  Estimated  posterior  means  for  odds  ratios  (ORs)  are  reported  with  their 
corresponding  95%  confidence  sets  (CS).  Confidence  sets  in  Bayesian  statistics  are 
similar  in  interpretation  to  confidence  levels  in  classical  statistics.  In  addition  to  the 
staffing  measures  of  interest,  regression  covariates  included  hospital  size,  shift  (day, 
evening,  night),  daily  acuity,  year,  and  daily  census.  For  comparability  and  simplicity  of 
presentation,  we  chose  to  fit  and  report  an  identical  model  specification  to  all  outcomes 
and  across  all  unit  types.  There  was  no  adjustment  for  multiple  testing  in  this  analysis. 
Unit  types  were  analyzed  separately. 

Aim  #2:  Explore  the  effects  of  patient  turnover  and  census  on  the 
relationship  between  structural  variables  and  outcomes. 

The  effect  of  census  was  included  in  the  initial  analysis;  however,  the  ADT 
variable  was  not  used.  The  turnover  variable  had  approximately  41  %  missing 
data  and  would  have  taken  additional  extensive  analyses  to  determine  the 
missing  data  mechanism  (missing  at  random  or  not  at  random),  and  therefore, 
whether  or  not  the  ADT  variable  could  be  imputed  and  with  what  imputation 
method.  Because  we  did  not  know  at  the  beginning  of  this  project  what  variables 
we  would  actually  analyze  (since  that  depended  on  the  reliability  and  validity 
analysis  in  MilNOD  III,  the  previous  study),  it  was  determined  that  this  particular 
variable  would  be  one  we  would  not  include  in  the  models  due  to  the  extent  of 
missing  data. 

Aim  #3:  Over  time,  have  MilNOD  participating  facilities  decreased  their  use  of 
restraints? 

This  aim  was  analyzed  with  the  same  type  of  Bayesian  HLM  model  as  previously 
explained  in  Aim  #1 .  The  model  was  specified  using  repeated  measurements  of 
restraint  prevalence  for  the  unit  quarterly  studies.  Data  were  weighted  by  the 
number  of  patients  in  each  prevalence  survey.  Unit  types  were  analyzed 
separately. 

Aim  #4:  Controlling  for  unit  type,  are  structural  variables  measured  on  day 
of  observed  restraint  assessment  associated  with  restraint  prevalence? 

Repeated  measurements  of  restraint  prevalence  for  unit  quarterly  studies  was  used  to 
specify  the  models.  Data  were  weighted  by  the  number  of  patients  included  in  the 
survey.  Compound  symmetry  covariance  structure  was  used  to  represent  equal 
correlation  in  all  units  for  study  outcomes  within  the  same  unit. 

Aim  #5:  Controlling  for  unit  type,  are  structural  variables  associated  with 
pressure  ulcer  prevalence  (hospital-acquired  stage  II  [HAPU2]  and  greater)? 
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For  this  analysis,  the  models  were  specified  using  repeated  measurements  of  HAPU2 
prevalence  for  the  unit  quarterly  studies.  Data  were  weighted  by  the  number  of  patients 
in  each  prevalence  survey.  This  aim  was  analyzed  with  the  same  type  of  Bayesian 
HLM  model  as  previously  explained  in  Aim  #1 .  Compound  symmetry  covariance 
structure  was  used  to  represent  equal  correlation  in  all  units  for  study  outcomes  within 
the  same  unit. 

The  same  number  of  units  and  quarters  were  used  in  both  the  restraint  and  the 
pressure  ulcer  analyses,  since  these  two  prevalence  studies  were  conducted  together. 

Aim  #6:  Controlling  for  patient  risk  (Braden  score,  BUN,  Creatinine, 

Albumin),  determine  which  units  have  a  greater  incidence  of  hospital- 
acquired  pressure  ulcers  stage  II  or  greater. 

and 

Aim  #7:  What  variables,  if  any,  predict  good  versus  poor  performance 
related  to  pressure  ulcer  prevention?  Good  performance  is  indicated  by 
having  high  risk  and  low  pressure  ulcer  prevalence.  Poor  performance  is 
defined  as  low  risk  and  high  pressure  ulcer  prevalence. 

Units  with  Braden  scores  <  16  and  HAPUs  <10,  were  classified  as  good 
performers,  since  this  indicates  an  at-risk  Braden  score  but  a  low  prevalence  of 
HAPU2s.  Poor  performers  were  classified  as  having  Braden  scores  of  >16  and  HAPU2 
prevalence  of  >10%.  Structural  variables  were  compared  between  the  two  groups. 

Most  good  and  poor  performing  units  were  critical  care.  Therefore,  the  analyses  were 
restricted  to  comparing  structural  variables  within  critical  care  units  only.  To  remove 
variation  due  to  a  small  number  of  patients  surveyed,  comparisons  were  further 
restricted  to  those  critical  care  units  in  which  5  patients  or  more  were  surveyed.  T-tests 
were  used  to  compare  structural  variables  between  the  3  "good"  performers  and  the  7 
"poor"  performers. 

Aim  #8:  Which  variables  predict  patient’s  report  of  satisfaction  (overall  and 
various  aspects)? 

Patient  satisfaction  was  measured  with  the  Patient  Satisfaction  with  Nursing  Care 
Questionnaire  (Jacox,  Bausell,  &  Mahrenholz,  1997).  Response  rates  were  previously 
discussed.  The  instrument  measures  three  dimensions  of  satisfaction:  satisfaction  with 
technical  skills  of  the  nurse,  satisfaction  with  caring,  and  satisfaction  with  teaching  about 
care  after  discharge.  In  order  to  provide  comparisons  to  what  CalNOC  measured, 
individual  items  were  also  examined.  The  scale  for  this  instrument  is  1  to  7,  with  higher 
numbers  indicating  more  satisfaction.  Our  plan  was  to  analyze  this  variable  using 
hierarchical  linear  modeling. 

Aim  #9:  Does  staffing  and  staff  category  impact  how  nursing 
personnel  respond  when  surveyed  about  the  work  environment  and 
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nursing  job  satisfaction? 


The  nurse  survey  data  set  which  included  job  satisfaction  and  work  environment 
variables  was  merged  with  the  staffing  and  patient  turnover  data  set  as  follows.  For 
each  individual  responding  to  the  survey,  the  staffing  and  patient  turnover  data  for  their 
particular  unit  was  aggregated  to  the  month  that  their  survey  was  returned.  For 
example,  a  survey  returned  in  June  of  2006  was  merged  with  the  staffing  data  for  that 
unit  for  the  month  of  June  2006.  Thus  every  case  (i.e.,  survey)  was  populated  with  the 
staffing  variables  aggregated  to  the  month  that  the  survey  was  returned.  SAS  version 
9.2  was  used  to  analyze  the  data. 

Descriptive  statistics  summarized  the  sample  characteristics.  Variables  were  screened 
for  distribution  and  collinearity  before  constructing  regression  models.  The  outcome 
variable,  job  satisfaction,  originally  a  five  category  variable  was  dichotomized  into  3  or  3 
and  4  =  “satisfied”  and  1  and  2  =  “dissatisfied”.  Generalized  linear  mixed  model 
analysis  was  used  with  a  binomial  distribution  and  logit  link  function.  The  models 
included  a  covariance  structure  that  accounted  for  the  clustering  of  nurses  within  units. 
The  following  staffing  variables  were  then  added  to  the  model:  RN  skill  mix,  military  mix, 
and  total  nursing  care  hours,  and  patient  turnover  rate  (admission,  transfers,  and 
discharges/census).  The  Generalized  Linear  Mixed  Models  used  in  the  analysis  were  fit 
by  maximum  likelihood  methods.  Goodness  of  fit  for  the  final  model  was  assessed  with 
a  chi-square  likelihood  ratio  test. 

Actual  analysis  of  the  data  was  conducted  at  both  the  University  of  San  Francisco  by 
Dr.  Moshe  Fridman  (for  the  first  seven  aims)  and  at  The  University  of  Alabama  at 
Birmingham  by  COL  (ret)  Patrician  in  conjunction  with  analyst,  Dr.  Andres  Azuero  for 
the  final  three  aims. 


Results 

Expansion  Aim:  Expand  the  number  of  participating  MilNOD  military  treatment 
facilities  (MTF)  from  seven  to  fourteen.  These  would  include  MTFs  representing 
all  three  branches  of  the  military  (Army,  Navy  and  Air  Force). 

The  initial  aim  to  include  fourteen  sites  representing  all  three  branches  of  the  military 
had  to  be  modified  during  the  study  due  to  facility  closures  and  realignment,  as  well  as 
inability  to  get  IRB  approval  at  one  site.  Thirteen  facilities,  of  varying  sizes  and 
TRICARE  regions,  ultimately  enrolled  and  participated.  These  facilities  did,  however, 
represent  all  three  branches  of  the  military  as  well  as  small  and  large  facilities. 
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Table  8 

Final  Status  of  MilNOD  Data  Collection  from  Participating  MTFs  During  MilNOD  lll/IV 


Military  Treatment  Size 
(Location) 

Service 

Size 

Data  Collection 
Period3 

N 

Days 

N 

Shifts 

Walter  Reed  AMC 
(Washington,  DC) 

Army 

Large 

Jul  03-Jun  06 

1095 

3285 

Madigan  AMC 
(Tacoma,  WA) 

Army 

Large 

Jul  03-Jun  06 

6727 

20181 

"Brooke  AMC 

(Fort  Sam  Houston,  TX) 

Army 

Large 

Dec  04-Jun  06 

5589 

16767 

"Wilford  Hall  Medical 

Center 

(Lackland  AFB,  TX) 

Air  Force 

Large 

Jul  05-Jun  06 

1834 

5502 

"National  NMC 
(Bethesda,  MD) 

Navy 

Large 

Oct  05-Jun  06 

1360 

4080 

"NMC  San  Diego 
(San  Diego,  CA) 

Navy 

Large 

Jan  06-Jun  06 

900 

2700 

Womack  AMC 
(Fort  Bragg,  NC) 

Army 

Medium 

Oct  03-Jun  06 

3285 

9855 

Malcolm  Grow  Medical 
Center 

(Andrews  AFB,  MD) 

Air  Force 

Small 

Oct  03-Mar  05 

1815 

5445 

"Naval  Hospital  Bremerton 
(Bremerton,  WA) 

Navy 

Small 

Sep  05-Jun  06 

561 

1683 

DeWitt  ACH 
(Fort  Belvoir,  VA) 

Army 

Small 

Oct  03-Mar  06 

2008 

6024 

"Naval  Hospital  Oak 

Harbor,  (Whidbey  Island, 
WA) 

Navy 

Small 

Sep  05-Jun  06 

277 

831 

"Bassett  ACH 
(Fort  Wainwright,  AK) 

Army 

Small 

Jun  04-Jun  06 

759 

2277 

"3rd  Medical  Group 
(Elmendorf  AFB,  AK) 

Air  Force 

Small 

Jul  05-Jun  06 

728 

2184 

TOTALS 

75,751 

227,253 

Note:  AMC  =  Army  Medical  Center;  ACH  =  Army  Community  Hospital;  AFB  =  Air  Force  Base;  NMC  = 
Naval  Medical  Center 

aDoes  not  include  data  run-in  period,  indicates  facility  new  in  MilNOD  IV. 


Analysis  Aim  #1 :  For  each  unit  type  (medical,  surgical,  ICU,  stepdown)  and 
controlling  for  hospital  size  (small,  large),  do  structural  variables  affect 
outcomes  at  the  shift  level? 


36 


Col  (ret)  Patrician  A.  Patrician 


Proposal  No  N03-P07 


Table  9  provides  the  shift-level  covariates  summarized  by  unit  type,  as  well  as  number 
of  shifts  that  are  included  in  the  analyses.  The  table  clearly  demonstrates  a  progression 
from  least  to  most  acute  patients  in  terms  of  nursing  care  hours  and  skill,  as  one  would 
expect  moving  from  medical  surgical  units  to  critical  care.  Figure  1  shows  the  observed 
rate  of  each  adverse  event  by  unit  type.  The  rates  of  all  adverse  events  are  low  when 
viewed  from  the  shift  level.  Medication  administration  errors  occurred  more  frequently 
than  did  falls. 


Table  9 

Shift  Level  Covariates  by  Unit  Type 


Variable 

Medical-Surgical 
(N=57,913  shifts) 

Step-Down 
(N=1 8,039  shifts) 

Critical  Care 
(N=35,570  shifts) 

Shift  census 

15.68  +  7.18 

10.63  +  5.54 

5.82  +  2.87 

Provider  type  by  proportion  of  total  hours 

%  RN 

51  +  14 

0.58  +  0.17 

0.77  +  0.19 

%  LPN 

0.22  +  0.17 

0.24  +  0.18 

0.14  +  0.17 

%  NA 

0.28  +  0.15 

0.19  +  0.15 

0.09  +  0.17 

Provider  category  by  percent  of  total  hours 

%  Active  military 

0.44  +  0.28 

0.36  +  0.24 

0.41  +  0.32 

%  DoD  civilian 

0.34  +  0.24 

0.39  +  0.26 

0.47  +  0.31 

%  Contract 

0.19  +  0.19 

0.22  +  0.19 

0.08  +  0.14 

%  Reserve 

0.027  +  0.09 

0.03  +  0.08 

0.05  +  0.11 

Nursing  care  hours  per  patient  shift  (NCH  PPS) 

Total 

4.29  +  2.84 

5.43  +  2.97 

9.42  +  6.27 

Licensed 

3.02  +  1.99 

4.38  +  2.46 

7.99  +  4.08 

RN 

2.15  +  1.60 

3.16  +  2.09 

6.87  +  3.90 

LPN 

0.87  +  0.96 

1.22  +  1.12 

1.12  +  1.70 

#  Patients  per  RN 

4.82  +  2.37 

3.26  +  1.59 

1.46  +  0.73 

Note:  mean  +  SD  reported;  N=1 1 1 ,552  shifts  with  complete  staffing  data 
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Figure  1 . 

Rates  of  Outcomes  by  Unit  Type* 


*  Rates  are  calculated  based  upon  percents  of  shifts  with  the  event  occurrence.  Of  99,412  shifts 
with  complete  data,  974  had  a  fall.  Of  99,338  shifts  with  complete  data,  21 1  falls  occurred  that 
resulted  in  injury.  Of  97,655  shifts  with  medication  administration  error  data,  1,395  had  a 
documented  medication  error. 

**MAE,  Medication  administration  errors. 


Falls  and  Falls  with  Injury 

Tables  10  and  1 1  show  the  results  of  analyses  for  falls  and  falls  with  injury,  respectively. 
A  greater  proportion  of  RNs  relative  to  unlicensed  assistive  personnel  (the  comparison 
category)  (higher  skill  mix)  was  significantly  associated  with  fewer  falls  in  medical- 
surgical  and  critical  care  units  but  not  in  step-down  units.  Fewer  falls  were  associated 
with  a  higher  percentage  of  DoD  civilian  nurses  working  on  a  shift.  A  greater  number  of 
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nursing  care  hours  per  patient  per  shift  was  significantly  associated  with  a  decreased 
likelihood  of  both  falls  and  falls  with  injury.  Increased  acuity  was  associated  with 
increased  falls,  although  this  finding  was  statistically  significant  for  medical-surgical 
units  only.  Increased  patient  census  was  significantly  related  to  falls  in  both  step-down 
and  medical-surgical  units.  More  falls  (but  not  more  of  those  with  injury)  occurred  on 
the  night  shift,  but  day  of  the  week  was  not  significantly  associated  with  falls.  Falls 
were  less  prevalent  in  the  final  years  of  the  study. 

Table  10 


Predictor 

Medical-Surgical 
OR  (95%  CS) 

Step-Down 

OR  (95%  CS) 

Critical  Care 

OR  (95%  CS) 

Shift  level 

Evening  shift 

1.14  (0.94-1.37) 

0.82  (0.55-1.19) 

0.98  (0.57-1.57) 

Night  shift 

1.36  (1.12-1.63) 

1.13  (0.77-1.61) 

0.82  (0.45-1.34) 

Skill  mix  (10%  decrease) 

%  RN 

1.11  (1.00-1.21) 

1.06  (0.95-1.18) 

1.20  (1.07-1.39) 

%  LPN 

1.08  (1.00-1.17) 

1.05  (0.94-1.16) 

1.03  (0.90-1.20) 

Staff  category  (10% 
decrease) 

%  Military 

1.09  (1.03-1.17) 

1.09  (1.00-1.19) 

1.14(1.01-1.28) 

%  Civilian 

1.48  (1.20-1.81) 

1.33  (1.11-1.51) 

1.36  (1.00-1.73) 

%  Contract 

1.17(1.10-1.26) 

1.05  (0.94-1.16) 

1.13  (0.98-1.30) 

Total  NCHPPS  (1  hour 

1.07  (1.01-1.14) 

1.14(1.03-1.28) 

1.11  (1.02-1.23) 

decrease) 

Day  level 

Day  of  week:  Monday 

1.24  (0.99-1.55) 

0.79  (0.52-1.24) 

0.80  (0.42-1.68) 

Census  (increase  of  3 

1.17(1.11-1.27) 

1.29  (1.12-1.40) 

1.00  (0.89-1.12) 

patients) 

Acuity  (1SD  increase) 

1.13  (1.02-1.27) 

1.00  (0.85-1.23) 

0.98  (0.72-1.31) 

Year:  2003 

1.32  (0.94-1.80) 

1.18  (0.55-1.92) 

0.80  (0.35-1.49) 

2004 

1.58  (1.22-2.01) 

1.47  (0.75-2.31) 

0.71  (0.30-1.27) 

2005 

1.20  (0.90-1.50) 

0.90  (0.55-1.41) 

1.09  (0.52-2.41) 

Unit  level:  Large  hospital 

2.34(1.18-4.19) 

6.77  (2.59- 
12.59) 

Hierarchical  Logistic  Regression  Modeling  Results  for  Falls* 

*  Odds  ratios  are  reported  with  their  corresponding  95%  confidence  sets  (CS;  similar  in 
interpretation  to  confidence  intervals,  but  conceptually  different  in  that  they  are  derived  from 
conditional  distribution  of  the  parameter  given  the  data  as  opposed  to  conditional  distribution  of 
the  data  given  the  parameter  in  classical  methods). 

RN,  registered  nurse;  LPN,  licensed  practical  nurse;  UAP,  unlicensed  assistive  personnel; 
NCHPPS,  nursing  care  hours  per  patient  shift;  OR,  odds  ratio;  CS,  confidence  set.  Reference 
group  excluded  for  each  categorical  variable:  Day  shift  for  shift  time,  UAP  for  provider  type, 
Reserve  for  provider  category,  2006  for  year,  Tuesday-Sunday  for  day  of  week,  and  Small 
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hospital  for  hospital  size.  Bolding  indicated  the  95%  CS  excludes  the  value  of  1.  P  values  are 
not  applicable  in  Bayesian  methods.  Only  large  hospitals  in  the  sample  had  step-down  units. 


Predictor 

Medical-Surgical 
OR  (95%  CS) 

Step-Down 

OR  (95%  CS) 

Critical  Care 

OR  (95%  CS) 

Shift  level 

Evening  Shift 

1.09  (0.72-1.62) 

0.80  (0.37-1.53) 

1.17  (0.38-2.83) 

Night  Shift 

1.24  (0.80-1.82) 

0.71  (0.32-1.36) 

1.11  (0.40-2.51) 

Skill  mix  (10% 
decrease) 

%  RN 

1.30  (1.17-1.49) 

1.05  (0.91-1.23) 

1.36  (1.18-1.55) 

%  LPN 

1.18  (0.10-1.33) 

1.03  (0.88-1.20) 

1.06  (0.90-1.23) 

Staff  category  (10% 
decrease) 

%  Military 

1.14  (1.04-1.27) 

1.10  (0.96-1.26) 

1.12  (0.98-1.28) 

%  Civilian 

1.48  (1.11-1.76) 

1.50  (1.09-1.90) 

1.27  (0.96-1.46) 

%  Contract 

1.18  (1.08-1.30) 

1.07  (0.92-1.25) 

1.14  (0.96-1.34) 

Total  NCHPPS  (1  hour 

1.15  (1.02-1.33) 

1.25  (1.00-1.67) 

1.51  (1.22-1.86) 

decrease) 

Day  level 

Day  of  week:  Monday 

0.76  (0.56-0.71) 

0.95  (0.27-1.94) 

1.44  (0.93-2.37) 

Census  (increase  of  3 

1.03  (0.95-1.14) 

1.57  (1.34-2.01) 

1.04  (0.79-1.15) 

patients) 

Acuity  (1SD  increase) 

1.09  (0.89-1.22) 

0.80  (0.40-1.52) 

1.07  (0.76-1.99) 

Year:  2003 

2.43  (1.24-4.74) 

1.51  (0.70-3.17) 

1.54  (1.08-4.69) 

2004 

3.32  (2.22-5.51) 

1.34  (0.63-2.48) 

1.97  (0.51-3.92) 

2005 

1.78  (0.97-3.00) 

0.49  (0.16-1.06) 

0.74  (0.26-3.62) 

Unit  level:  Large 

7.19  (2.32-17.31) 

20.16  (4.78-45.74) 

hospital 

Table  1 1 


Hierarchical  Logistic  Regression  Modeling  Results  for  Falls  with  Injury* 

*  RN,  registered  nurse;  LPN,  licensed  practical  nurse;  UAP,  unlicensed  assistive  personnel; 
NCHPPS,  nursing  care  hours  per  patient  shift;  OR,  odds  ratio;  CS,  confidence  set.  Reference 
group  excluded  for  each  categorical  variable:  Day  shift  for  shift  time,  UAP  for  provider  type, 
Reserve  for  provider  category,  2006  for  year,  Tuesday-Sunday  for  day  of  week,  and  Small 
hospital  for  hospital  size.  Bolding  indicated  the  95%  CS  excludes  the  value  of  1.  P  values  are 
not  applicable  in  Bayesian  methods.  Only  large  hospitals  in  the  sample  had  step-down  units. 


Medication  Errors 


Results  of  the  analysis  of  medication  administration  errors  are  depicted  in  Table  12.  As 
with  falls,  an  increased  proportion  of  RNs  on  a  shift  was  significantly  associated  with 
fewer  medication  administration  errors  in  medical-surgical  and  critical  care  units.  A 
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higher  proportion  of  DoD  civilian  nursing  staff  was  associated  with  fewer  medication 
administration  errors,  particularly  in  step-down  and  critical  care  units. 

A  higher  number  of  total  nursing  care  hours  per  shift  was  significantly  associated  with  a 
decrease  in  the  likelihood  of  a  medication  error  occurring  on  a  shift  in  medical-surgical 
and  critical  care  units.  Night  shift  had  significantly  fewer  medication  administration  error 
occurrences.  An  increased  daily  census  and  increased  acuity  were  significantly 


Predictor 

Medical-Surgical 
OR  (95%  CS) 

Step-Down 

OR  (95%  CS) 

Critical  Care 

OR  (95%  CS) 

Shift  level 

Evening  Shift 

0.97  (0.83-1.14) 

0.78  (0.56-1.06) 

0.90  (0.64-1.22) 

Night  Shift 

0.43  (0.36-0.52) 

0.46  (0.31-0.66) 

0.41  (0.27-0.59) 

Skill  mix  (10%  decrease) 

%  RN 

1.13  (1.04-1.23) 

1.03  (0.91-1.19) 

1.17(1.06-1.35) 

%  LPN 

1.10  (1.01-1.20) 

0.95  (0.85-1.10) 

1.05  (0.94-1.20) 

Staff  category  (10% 
decrease) 

%  Military 

1.07  (0.98-1.20) 

0.95  (0.84-1.10) 

1.08  (0.97-1.20) 

%  Civilian 

1.38  (0.98-1.87) 

1.67  (1.09-2.39) 

1.47(1.00-2.01) 

%  Contract 

1.06  (0.96-1.18) 

0.96  (0.84-1.12) 

1.03  (0.90-1.17) 

Total  NCHPPS  (1  hour 

1.13  (1.06-1.21) 

1.02  (0.95-1.12) 

1.05  (1.01-1.10) 

decrease) 

Day  level 

Day  of  week:  Monday 

0.82  (0.64-1.01) 

1.09  (0.69-1.66) 

1.00  (0.59-1.57) 

Census  (increase  of  3 

1.07(1.00-1.15) 

1.36(1.10-1.60) 

1.14(1.04-1.25) 

patients) 

Acuity  (1SD  increase) 

1.13  (1.01-1.25) 

0.97  (0.70-1.23) 

0.90  (0.71-1.22) 

Year:  2003 

1.19  (0.96-1.54) 

1.27  (0.65-2.27) 

0.46  (0.23-0.92) 

2004 

1.17(0.90-1.50) 

2.13  (1.25-3.55) 

0.87  (0.49-1.70) 

2005 

0.81  (0.64-1.03) 

1.08  (0.59-1.84) 

0.61  (0.33-0.99) 

Unit  level:  Large  hospital 

1.34  (0.56-2.87) 

1.17(0.62-3.97) 

Table  12 


Hierarchical  Logistic  Regression  Modeling  Results  for  Medication  Errors* 

*  RN,  registered  nurse;  LPN,  licensed  practical  nurse;  UAP,  unlicensed  assistive  personnel; 
NCHPPS,  nursing  care  hours  per  patient  shift;  OR,  odds  ratio;  CS,  confidence  set.  Reference 
group  excluded  for  each  categorical  variable:  Day  shift  for  shift  time,  UAP  for  provider  type, 
Reserve  for  provider  category,  2006  for  year,  Tuesday-Sunday  for  day  of  week,  and  Small 
hospital  for  hospital  size.  Bolding  indicated  the  95%  CS  excludes  the  value  of  1.  P  values  are 
not  applicable  in  Bayesian  methods.  Only  large  hospitals  in  the  sample  had  step-down  units. 
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associated  with  more  medication  administration  errors.  Annual  trends  varied,  but 
medication  errors  were  generally  lower  in  subsequent  study  years. 


Needlestick  Injuries 

Needlestick  injuries  had  some  associations  with  staffing  as  depicted  in  Table  13. 
Needlesticks  were  associated  with  shifts  that  had  lower  RN  skill  mix,  a  fewer  proportion 
of  civilian  nurses,  and  decreased  nursing  care  hours.  Patient  acuity  was  not  associated 
with  the  odds  of  a  needlestick  injury  on  a  shift. 

Table  13 

Hierarchical  Logistic  Regression  Modeling  Results  for  Needlestick  Injuries * 


Predictor 

Medical-Surgical 
OR  (95%  CS) 

Step-Down 

OR  (95%  CS) 

Critical  Care 

OR  (95%  CS) 

Shift  level 

Evening 

0.64  (0.31-1.13) 

0.91  (0.31-2.06) 

0.83  (0.34-1.69) 

Night 

0.58  (0.28-1.05) 

0.72  (0.22-1.70) 

0.45  (0.16-0.99) 

Provider  skill  mix  (10%  decreased  effect) 

%  RN 

1.32  (1.14-1.54) 

1.38  (1.18-1.60) 

1.35  (1.18-1.57) 

%  LPN 

1.07  (0.92-1.27) 

1.24  (1.06-1.45) 

1.14  (0.96-1.36) 

Provider  category  (10%  decreased  effect) 

%  Military 

1.10  (0.98-1.25) 

1.24  (1.06-1.45) 

1.07  (0.94-1.22) 

%  Civilian 

1.54  (1.00-2.12) 

1.22  (1.05-1.44) 

1.14  (0.83-1.31) 

%  Contract 

1.16  (0.99-1.36) 

1.21  (1.03-1.42) 

1.05  (0.89-1.25 

Total  NCH  PPS  (1  hour 
decrease) 

1.43  (1.15-1.89) 

0.97  (0.90-1.08) 

1.52  (1.28-1.89) 

Day  level 

Day  of  week:  Monday 

0.77  (0.32-1.78) 

1.06  (0.56-1.93) 

1.80  (0.54-4.41) 

Census 

(increase  of  3  patients) 

0.93  (0.77-1.06) 

0.83  (0.61-0.99) 

1.08  (0.93-1.16) 

Acuity  (1SD  increase) 

1.18  (0.88-1.55) 

0.92  (0.59-1.31) 

1.04  (0.62-1.66) 

Year:  2003 

1.37  (0.20-6.60) 

** 

** 

2004 

2.15  (0.48-11.28) 

** 

** 

2005 

0.99  (0.13-6.86) 

** 

** 

Unit  level:  Large  hospital 

1.66  (0.60-4.08) 

a 

12.54  (2.46- 
38.90) 

*NCHPPS  =  nursing  care  hours  per  patient  shift.  OR=Odds  ratio;  CS=Confidence  Set. 
Reference  group  excluded  for  each  categorical  variable:  unlicensed  assistive  personnel  for 
provider  type,  Reserve  for  provider  category,  day  shift  for  shift  time,  Tuesday  to  Sunday  for  day 
of  week. 

Bolding  indicated  the  95%  CS  excludes  the  value  of  1 .  P  values  are  not  applicable  in  Bayesian 
methods.  aOnly  large  hospitals  in  the  sample  had  step-down  units. 
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All  Adverse  Events  Combined 

All  adverse  events  combined  were  analyzed  to  look  at  overall  effects  of  staffing 
variables.  Results  are  presented  in  Table  14  below. 


Table  14 

Hierarchical  Logistic  Regression  Modeling  Results  for  Any  Adverse  Occurrences 


Predictor 

Medical-Surgical 

Step-Down 

Critical  Care 

Shift  time 

Evening 

1.01  (0.89-1.15) 

0.83  (0.65-1.04) 

0.93  (0.71-1.20) 

Night 

0.75  (0.65-0.85) 

0.76  (0.57-0.97) 

0.51  (0.37-0.68) 

Provider  skill  mix  (10%  decrease) 

%  RN 

1.07  (1.00-1.16) 

1.07  (0.98-1.19) 

1.10  (0.95-1.30) 

%  LPN 

1.03  (0.96-1.13) 

1.03  (0.93-1.15) 

1.02  (0.90-1.16) 

Provider  category  (10%  decrease) 

%  Military 

1.04  (0.97-1.13) 

1.06  (0.96-1.19) 

1.06  (0.95-1.18) 

%  Civilian 

1.42  (1.10-1.79) 

1.29  (0.99-1.61) 

1.45  (1.00-2.03) 

%  Contract 

1.04  (0.97-1.13) 

1.04  (0.93-1.18) 

1.05  (0.93-1.18) 

Total  NCH  PPS 

(decrease  by  one  hour) 

1.08  (1.03-1.14) 

1.01  (0.97-1.07) 

1.05  (1.01-1.09) 

Day  level 

Day  of  week:  Monday 

1.00  (0.85-1.16) 

0.87  (0.63-1.20) 

1.94  (0.65-1.34) 

Census  (increase  of  3 
patients) 

1.13  (1.08-1.18) 

1.27  (1.13-1.44) 

1.09  (1.01-1.18) 

Acuity  (1SD  increase) 

1.13  (1.04-1.23) 

1.03  (0.84-1.25) 

1.01  (0.80-1.25) 

Year:  2003 

1.50  (1.10-1.86) 

1.55  (0.94-2.56) 

0.80  (0.42-1.38) 

2004 

1.64  (1.37-1.95) 

2.38  (1.57-3.52) 

1.31  (0.74-2.07) 

2005 

1.20  (1.01-1.42) 

1.41  (1.01-2.07) 

1.13  (0.68-1.67) 

Unit  level:  Large  hospital 

0.70  (0.23-1.65) 

a 

1.57  (0.38-4.77) 

*NCHPPS  =  nursing  care  hours  per  patient  shift.  OR=Odds  ratio;  CS=Confidence  Set. 
Reference  group  excluded  for  each  categorical  variable:  unlicensed  assistive  personnel  for 
provider  type,  Reserve  for  provider  category,  day  shift  for  shift  time,  Tuesday  to  Sunday  for  day 
of  week. 

Bolding  indicated  the  95%  CS  excludes  the  value  of  1 .  P  values  are  not  applicable  in  Bayesian 
methods.  aOnly  large  hospitals  in  the  sample  had  step-down  units. 


Aim  #2:  Explore  the  effects  of  patient  turnover  and  census  on  the 
relationship  between  structural  variables  and  outcomes. 
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The  effect  of  census  is  included  in  the  above  analysis;  however,  the  ADT  variable 
was  not  used.  The  turnover  variable  had  approximately  41  %  missing  data. 
Additional  analyses  are  necessary  to  determine  whether  or  not  the  missing  data 
was  missing  at  random  or  not  at  random,  and  therefore,  whether  or  not  the  ADT 
variable  could  be  imputed  and  with  what  imputation  method.  Because  we  did  not 
know  at  the  beginning  of  this  project  what  variables  we  would  actually  analyze 
(since  that  depended  on  the  reliability  and  validity  analysis  in  MilNOD  III,  the 
previous  study),  it  was  determined  that  this  particular  variable  would  be  one  we 
could  not  include  in  the  models  due  to  the  extent  of  missing  data. 

A  higher  census  was  associated  with  lower  needlestick  injuries  and  lower 
medication  errors  and  higher  falls  with  injuries  on  in  step  down  units,  as  well  as 
higher  adverse  events  overall  in  all  three  unit  types.  Step  down  units  appear  to 
be  more  sensitive  to  differences  in  census. 

Aim  #3:  Over  time  have  MilNOD  participating  facilities  decreased  their  use  of 
restraints? 

A  total  of  54  units  were  included  in  the  analyses;  however  because  of  the 
staggered  nature  of  site  participation,  the  same  units  were  not  included  every 
year.  A  total  of  132  quarters  were  analyzed,  representing  the  54  participating 
units.  Table  15  lists  the  observed  restraint  rates  (weighted  by  patient  numbers) 
by  unit  type  per  year. 

Table  15 

Observed  Restraint  Rates 


Restraint  Prevalence 

2003 

2004 

2005 

2006 

Med-Surg 

4.14 

1.82 

1.09 

0.93 

Stepdown 

2.50 

6.06 

1.14 

2.90 

Critical  Care 

17.39 

38.89 

17.65 

32.26 

Small  hospitals  had  too  few  surveys  for  meaningful  analysis.  Medical-surgical 
units  has  a  downward  trend  in  restraint  prevalence,  but  this  was  not  statistically 
significant.  Critical  care  units  actually  increased  restraint  prevalence  over  the 
years  of  the  study  (Table  15). 


Table  16  shows  trend  p  values  for  changes  in  restraint  prevalence  over  time. 
Only  critical  care  unit  trends  were  statistically  significant,  but  in  the  wrong 
direction;  critical  care  units  increased  restraint  use  over  the  study  period. 
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Table  16 

Change  in  Restraint  Prevalence  over  Time 


Model 
results  by 
unit  type 

2003 

2004 

2005 

2006 

trend  p- 
value 

Medical-Surgical  Units:  29  units;  69  quarters 

A  in 

restraint 

prevalence 

0 

-2.15 

-2.91 

-2.81 

.292 

p  value 

- 

0.24 

0.07 

0.09 

Step  down  Units:  8  unit 

ts;  23  quarters 

A  in 

restraint 

prevalence 

0 

3.22 

-1.21 

1.61 

0.17 

p  value 

0.18 

0.54 

0.44 

Critical  Care  Units:  18  units;  43  quarters 

A  in 

restraint 

prevalence 

0 

24.17 

-6.05 

7.40 

0.03 

p  value 

- 

0.03 

0.50 

0.45 

Note:  Estimated  from  repeated  measurement  models  for  each  unit  type 
(unadjusted). 


Aim  #4:  Controlling  for  unit  type,  are  structural  variables  measured  on  day 
of  observed  restraint  assessment  associated  with  restraint  prevalence? 

Repeated  measurements  of  restraint  prevalence  for  unit  quarterly  studies  was 
used  to  specify  the  models.  Data  were  weighted  by  the  number  of  patients  included  the 
survey.  Compound  symmetry  covariance  structure  was  used  to  represent  equal 
correlation  in  all  units  for  study  outcomes  within  the  same  unit. 

None  of  the  following  variables  were  statistically  significant  in  repeated 
measurement  models  for  restraint  prevalence,  adjusting  for  hospital  size  and  unit  type: 
skill  mix,  staff  category,  NCHPPS,  RN  to  patient  ratio,  daily  census,  and  average  acuity. 

When  separate  models  by  unit  type  were  analyzed,  the  following  structural  variables 
were  statistically  significant  in  relationship  to  restraint  prevalence  (models  not  shown): 

1 .  A  10%  increase  in  civilian  staff  was  associated  with  a  0.7%  increase  in 
restraint  prevalence  in  step  down  units  only  (p=0.022). 
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2.  A  10%  increase  in  contract  staff  was  associated  with  a  5.8%  increased 
restraint  prevalence  in  critical  care  units  only  (p=0.040). 

3.  Each  additional  1  hour  of  RN  staffing  (1  RN-NCHPPS)  was  associated  with  a 
3.89%  decrease  in  restraint  prevalence  (p=0.007). 

Aim  #5:  Controlling  for  unit  type,  are  structural  variables  associated  with 
pressure  ulcer  prevalence  (hospital-acquired  stage  II  [HAPU2]  and  greater)? 

For  this  analysis,  the  models  were  specified  using  repeated  measurements  of  HAPU2 
prevalence  for  the  unit  quarterly  studies.  Data  were  weighted  by  the  number  of  patients 
in  each  prevalence  survey.  This  aim  was  analyzed  with  the  same  type  of  Bayesian 
HLM  model  as  previously  explained  in  Aim  #1 .  Compound  symmetry  covariance 
structure  was  used  to  represent  equal  correlation  in  all  units  for  study  outcomes  within 
the  same  unit. 

The  same  number  of  units  and  quarters  were  used  in  both  the  restraint  and  the 
pressure  ulcer  analyses,  since  these  two  prevalence  studies  were  conducted  together. 

The  major  findings  were: 

1 .  Critical  care  units  had  the  highest  rates  of  HAPU2,  followed  by  medical 
surgical  units  (see  Table  17). 

2.  Average  Braden  scores  were  stable  across  unit  types  and  years  (Table  18). 

3.  For  each  1  point  increase  in  Braden  scores,  there  was  an  increase  in  critical 
care  units  FIAPU2  rates  (by  11.15%,  p  =  0.003). 

4.  The  only  statistically  significant  staffing  finding  relative  to  HAPU2  prevalence 
was  that  a  10%  increase  in  civilian  staff  was  associated  with  a  14%  decrease  (p=0.05) 
in  HAPU2s  in  medical  surgical  units  only. 


Table  17 

HAPU2s  Prevalence  by  Unit  Type  over  Time 


2003 

2004 

2005 

2006 

Medical- 

Surgical 

6.51 

9.09 

6.25 

5.26 

Stepdown 

2.50 

3.03 

3.41 

0 

Critical  care 

17.39 

11.11 

18.82 

22.58 

Table  18 

Average  Braden  Score  by  Unit  Type  over  Time 


2003 

2004 

2005 

2006 

Medical- 

18.6 

18.1 

18.8 

18.3 
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Surgical 

Stepdown 

18.5 

18.1 

18.6 

18.3 

Critical  care 

17.3 

15.0 

17.1 

16.3 

Aim  #6:  Controlling  for  patient  risk  (Braden  score,  BUN,  Creatinine, 
Albumin),  determine  which  units  have  a  greater  incidence  of  hospital- 
acquired  pressure  ulcers  stage  II  or  greater. 

Even  when  controlling  for  higher  risk,  critical  care  units  had  the  highest  incidence 
of  HAPU2s.When  separating  the  units  into  good  versus  poor  performers,  1 
stepdown  and  5  critical  care  units  were  in  the  "good  category"  and  in  the  "poor 
category"  were  9  ICUs,  1  stepdown  and  5  medical-surgical  units.  See  Table  19. 


Aim  #7:  What  variables,  if  any,  predict  good  versus  poor  performance 
related  to  pressure  ulcer  prevention?  Good  performance  is  indicated  by 
having  high  risk  and  low  pressure  ulcer  prevalence.  Poor  performance  is 
defined  as  low  risk  and  high  pressure  ulcer  prevalence. 

Units  with  Braden  scores  <  16  and  HAPUs  <10,  were  classified  as  good 
performers,  since  this  indicates  an  at-risk  Braden  score  but  a  low  prevalence  of 
HAPU2s.  Poor  performers  were  classified  as  having  Braden  scores  of  >16  and 
HAPU2  prevalence  of  >10%.  Structural  variables  were  compared  between  the 
two  groups.  Most  good  and  poor  performing  units  were  critical  care.  Therefore, 
the  analyses  were  restricted  to  comparing  structural  variables  within  critical  care 
units  only.  To  remove  variation  due  to  a  small  number  of  patients  surveyed, 
comparisons  were  further  restricted  to  those  critical  care  units  in  which  5  patients 
or  more  were  surveyed.  T-tests  were  used  to  compare  structural  variables 
between  the  3  "good"  performers  and  the  7  "poor"  performers.  Table  19  presents 
the  statistically  significant  differences.  Good  performers  had  a  higher  contract 
mix  as  compared  to  poor  performers.  Good  performing  units  had  a  higher 
number  of  patients  per  RN  as  compared  to  poorly  performing  units.  Good 
performers  had  0%  reservists  and  poor  performers  averaged  6.5%  reserve  staff, 
but  this  was  not  statistically  significant.  Poor  performers  had  HAPU2  rates  10 
times  higher  than  good  performers. 

Table  19 

HAPU2  Prevalence:  Good  versus  Poor  Performing  Critical  Care  Units 


Variable 

Good 

Poor 

t-test 

performers 

performers 

p-value 

Contract  mix 

36.2% 

1 1 .5% 

0.01 

Pt:RNs 

1.97 

1.34 

0.04 
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Also,  restraint  prevalence  was  an  unexpected  finding  in  that  a  higher  restraint 
prevalence  was  associated  with  lower  HAPU2  prevalence  (not  shown  here). 


Aim  #8;  Which  variables  predict  patient’s  report  of  satisfaction  (overall  and 
various  aspects)? 

Patient  satisfaction  was  measured  with  the  Patient  Satisfaction  with  Nursing  Care 
Questionnaire  (Jacox,  Bausell  &  Mahrenholz,  1997).  The  instrument  measures 
three  dimensions  of  satisfaction:  satisfaction  with  technical  skills  of  the  nurse, 
satisfaction  with  caring,  and  satisfaction  with  teaching  about  care  after  discharge. 
In  order  to  provide  comparisons  to  what  CalNOC  measured,  individual  items 
were  also  examined.  The  scale  for  this  instrument  is  1  to  7,  with  higher  numbers 
indicating  more  satisfaction.  Table  20  shows  the  scores. 

Table  20 

Patient  Satisfaction  Scores 


Item  or  scale: 
Satisfaction  with: 

Mean 

(SD) 

Range 

Median 

Mode 

Caring 

5.98 

(1.44) 

1-7 

6.75 

7 

Technical  skills 

6.13 

(1.28) 

1-7 

6.75 

7 

Discharge 

Teaching 

5.68 

(1.62) 

1-7 

6.00 

7 

Nursing  care  in 
general 

5.93 

(1.36) 

1-7 

6.42 

7 

Pain 

management 

6.14 

(1.37) 

1-7 

7 

7 

Overall 

Satisfaction 

6.08 

(1.47) 

1-7 

7 

7 

Note:  N  =  1576-1721  surveys 


Patients  were  most  satisfied  with  pain  management  and  least  satisfied  with 
discharge  teaching.  A  one-way  ANOVA  showed  no  significant  differences  on 
satisfaction  ratings  between  hospitals  on  any  of  the  components.  A  mixed 
models  procedure  in  SAS  was  conducted  to  look  for  effects  over  time  (years)  and 
also  between  facilities.  No  significant  effects  were  discovered.  Because  there 
were  no  between-facility  differences  in  patient  satisfaction,  no  further  analysis 
could  be  conducted  to  evaluate  effects  of  structural  variables  on  patient 
satisfaction. 

Aim  #9:  Does  staffing  and  staff  category  impact  how  nursing 
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personnel  respond  when  surveyed  about  the  work  environment  and 
nursing  job  satisfaction? 

To  evaluate  this  aim,  the  Practice  Environment  Scale  (PES)  and  its  subscales  from 
nurse  survey  data  were  merged  with  shift  level  data.  Actual  staffing  data  from  the 
nurses’  unit  was  averaged  for  that  month.  The  aggregated  staffing  data  was  merged 
with  the  survey  data  to  reflect  average  staffing  that  was  experienced  during  the  same 
month  that  the  nurse  submitted  his/her  survey.  For  example,  a  survey  returned  in  June 
of  2006  was  merged  with  the  staffing  data  for  that  unit  for  the  month  of  June  2006. 

Thus  every  case  (i.e. ,  survey)  was  populated  with  the  staffing  variables  aggregated  to 
the  month  that  the  survey  was  returned.  A  total  of  1586  staff  members  completed  the 
surveys  (response  rate  35%),  with  997  providing  complete  responses  to  the  PES 
survey. 


We  first  analyzed  the  composite  PES  score.  Using  linear  mixed  models  that  accounted 
for  the  nesting  of  nurses  within  units,  we  found  that  higher  total  nursing  care  hours  and 
being  a  military  nurse  were  significantly  associated  with  the  report  of  a  more  favorable 
practice  environment  (see  Table  21).  The  Collegial  Nurse-Physician  Relations  (CNPR) 
subscale  had  a  significant  association  not  with  nursing  care  hours,  but  with  RN  skill  mix 
such  that  in  environments  with  a  higher  RN  skill  mix,  nurses  reported  a  more  favorable 
collaborative  relationship  with  physicians.  The  Staffing  and  Resource  Adequacy  (SRA) 
subscale  has  a  statistically  significant  inverse  association  with  being  an  RN  and  a 
positive  association  with  total  nursing  care  hours.  The  Nurse  Manager  Ability, 
Leadership,  and  Support  (NMALS),  Nursing  Foundations  for  Quality  Care  (NFQC),  and 
Nursing  Participation  in  Hospital  Affairs  (NPHA)  subscales  all  has  statistically  significant 
associations  with  being  a  military  nurse,  i.e.,  military  nurses  rated  these  subscales  as 
more  favorable. 

Table  21 


Summary  of  Practice  Environment  Scale  Results 


Outcome 

Associated  with 

Regression 

estimate 

Standard  Error 

t-value 

P 

PES 

Military  nurse 

.192 

.036 

5.40 

<.001 

Composite 

TNCHPPS 

.016 

.007 

2.10 

.040 

CNPR 

Military  nurse 

.094 

.039 

2.41 

.016 

RN  skill  mix 

.517 

.175 

2.95 

.005 

SRA 

RN 

-.143 

.057 

-2.53 

.012 

TNCHPPS 

.045 

.010 

4.39 

<.001 

NMALS 

Military  nurse 

.221 

.043 

5.17 

<.001 

NFQC 

Military  nurse 

.190 

.031 

6.13 

<.001 

NPHA 

Military  nurse 

.359 

.035 

10.33 

<.001 

Notes:  See  texl 

:  above  for  abbrevial 

tions.  Models  incl 

uded  RN  (yes/no),  military  (yes/no),  total 

experience  (years),  RN  skill  mix,  military  mix,  and  total  nursing  care  hours  per  patient  per  shift 
(TNCHPPS) 
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The  nurse  satisfaction  analysis  is  described  below. 

Aim  #10:  Does  patient  turnover  contribute  to  nurse  dissatisfaction? 

In  addition  to  the  merged  data  described  above,  patient  turnover  was  added  to  the 
models  predicting  nurse  satisfaction.  The  outcome  variable,  job  satisfaction,  originally  a 
five  category  variable  was  dichotomized  into  “satisfied”  and  “dissatisfied”.  Generalized 
linear  mixed  model  analysis  was  used  with  a  binomial  distribution  and  logit  link  function. 
The  models  included  a  covariance  structure  that  accounted  for  the  clustering  of  nurses 
within  units. 

A  total  of  1586  staff  members  completed  the  surveys  (response  rate  35%),  with  1438 
providing  responses  to  the  job  satisfaction  question.  The  mean  age  of  nursing 
personnel  was  37.3  with  a  SD  of  1 0.8.  Most  of  the  sample  (78.33%)  were  RNs.  Out  of 
those  that  identified  themselves  as  RNs,  46.7%  (N=776)  were  military  and  53.3% 
(N=680)  were  civilians.  Nursing  respondents  had  been  in  their  current  position  for  an 
average  of  4.1  years  with  a  total  of  9.26  years  of  nursing  experience  (N=1481).  A  five 
point  Likert  scale  ranging  from  1  =  "very  dissatisfied"  to  5  =  "very  satisfied"  was  used  for 
the  response  set.  Overall,  respondents  indicate  that  they  were  either  very  satisfied 
23.99%  or  were  satisfied  with  their  current  jobs  36.02%.  Few  were  dissatisfied 
(15.37%)  or  very  dissatisfied  (9.53%).  Overall,  the  nurse  respondents  were  generally 
satisfied  with  their  current  jobs. 

Among  the  independent  variables,  age  and  years  experience  were  highly  correlated 
(tolerance  =  0.37),  and  age  was  dropped  from  further  analyses.  First,  the  individual 
nurse-level  variables  were  entered  into  the  model  (provider  type,  category  and  years 
experience).  Both  being  an  RN  and  being  a  military  nurse  had  significant  associations 
with  satisfaction,  but  in  opposite  directions.  The  odds  of  being  satisfied  were 
significantly  higher  for  RNs  than  for  non-RNs  (OR  1 .46,  p  .04);  whereas  the  odds  of 
being  satisfied  among  military  personnel  were  significantly  lower  than  the  odds  for 
civilians  (OR  0.66,  p  .01 ).  Years  experience  had  no  effect  on  job  satisfaction. 

The  following  staffing  variables  were  then  added  to  the  model:  RN  skill  mix,  military  mix, 
and  total  nursing  care  hours,  and  patient  turnover  rate  (admission,  transfers,  and 
discharges/census).  None  of  the  staffing  variables  nor  patient  turnover  were  associated 
with  satisfaction,  and  thus  it  was  decided  to  remove  these  variables  from  the  model. 

Because  the  researchers  wanted  to  look  more  closely  at  the  RN  versus  military  effect, 
an  interaction  term  was  added  to  the  model.  The  final  model  demonstrates  a  significant 
effect  of  military  status,  as  well  as  a  statistically  significant  interaction  between  RN  and 
military  status,  such  that  although  military  personnel  overall  were  less  satisfied  with  their 
jobs  (OR=0.49,  p=0.0006),  military  personnel  who  were  also  RNs  were  significantly 
more  satisfied  than  military  non-RNs  (OR=2.18,  p=0.0 1 62). 


50 


Col  (ret)  Patrician  A.  Patrician 


Proposal  No  N03-P07 


The  researchers  used  the  final  model  to  calculate  predicted  probabilities,  odds,  and 
odds  ratios  to  compare  various  categories  of  personnel.  As  Table  22  shows,  among 
those  providers  who  were  LPN’s  and  unlicensed  personnel,  the  odds  of  being  satisfied 
for  civilians  were  nearly  three  times  the  odds  for  military  personnel.  Among  military 
providers,  the  odds  of  satisfaction  for  RNs  were  over  two  times  the  odds  for  LPN’s  and 
UAPs. 

The  Generalized  Linear  Mixed  Models  used  in  the  analysis  were  fit  by  maximum 
likelihood  methods.  Goodness  of  fit  for  the  final  model  was  assessed  with  a  chi-square 
likelihood  ratio  test.  The  difference  in  -2  log  likelihood  between  an  intercept-only  model 
and  the  final  model  (with  three  parameters  for  the  fixed  effects)  was  261 .9,  which  is 
highly  significant  with  three  degrees  of  freedom.  Dispersion  of  the  model  was  assessed 
with  the  estimated  dispersion  scale  parameter,  which,  at  a  value  of  0.93,  suggested  no 
considerable  evidence  of  over-  or  under-  dispersion.  Thus,  the  model  appeared  to 
appropriately  account  for  the  variability  in  the  data. 


Table  22 

Differences  in  Job  Satisfaction  by  Skill  Level  and  Provider  Category. 


Odds  Ratios 

Estimate 

95%  C.l. 

civil  non-RN 

vs 

civil  RN 

1.068 

0.65-  1.75 

civil  non-RN 

vs 

Mil  non-RN 

2.919 

1.58-5.39 

civil  non-RN 

vs 

Mil  RN 

1.335 

0.82-2.17 

Mil  RN 

vs 

Mil  non-RN 

2.186 

1.33-3.57 

Civilians 

vs 

Military 

2.020 

1.38-2.94 

RN 

vs 

non-RN 

1.353 

0.94-1.93 

Note:  Mil  =  milil 

:ary 

Discussion 

In  summary  our  findings  show  significant  differences  in  adverse  events  by  unit  type  and 
significant  associations  of  staffing  with  adverse  events,  supporting  the  assumption  that 
adverse  events  occur  during  shifts  that  are  staffed  with  fewer  personnel  overall,  and 
fewer  RNs  and  civilian  staff  in  particular.  Differences  in  fall  rates  by  unit  type  were 
expected  because  of  the  nature  of  patients’  mobility  restrictions  in  each  type  of  unit. 
Differences  in  rates  of  medication  administration  errors  were  also  expected  based  on 
the  observation  that  critical  care  nurses  care  for  fewer  patients  and  thus  may  be  more 
familiar  with  the  medication  regimens  for  each  patient,  whereas  medical-surgical  nurses 
care  for  more  and  may  not  be  as  familiar  with  each  patient’s  specific  medications. 
Needlestick  injuries  to  nurses  were  constant  across  all  unit  types. 
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Relationships  with  RN  skill  mix  were  most  notable  in  falls  with  injury;  each  10% 
decrease  in  RN  skill  mix  was  associated  with  a  36%  increase  in  the  likelihood  of  falls 
with  injury  in  critical  care  units  and  with  a  30%  increase  on  medical-surgical  units.  This 
association  was  greater  on  critical  care  units,  where  high  acuity  patients  are  beginning 
ambulation  after  critical  illness.  The  RNs  on  those  units  could  be  more  aware  of 
underlying  physiological  implications  of  patient-specific  medications,  extended  bed  rest, 
illness,  and  injury  matters  that  might  not  be  well  understood  by  other  levels  of  nursing 
personnel.  A  decrease  in  RN  skill  mix  was  also  associated  with  medication  errors  in 
medical  surgical  and  critical  care  units  and  needlestick  injuries  in  all  three  unit  types. 

There  was  a  strong  relationship  between  total  staffing  (nursing  care  hours  per  patient 
shift)  and  falls  with  injury.  Depending  on  unit  type,  a  15-51%  increase  in  falls  with  injury 
was  observed  with  each  decrease  of  1  hour  of  nursing  care  per  shift.  This  finding  differs 
from  a  CALNOC  report  showing  that  staffing  changes  from  mandated  nurse-  to-patient 
ratios  were  not  associated  with  patient  falls  (Donaldson,  Bolton,  Aydin,  Brown,  Elashoff, 
&  Sandhu,  2005).  One  explanation  may  be  the  relatively  wide  variation  in  staffing  noted 
across  shifts  in  our  study.  This  variation  is  likely  muted  when  staffing  data  are 
aggregated  and  analyzed  at  a  monthly  level,  the  procedure  commonly  used  with  other 
nursing  outcomes  databases.  A  smaller  but  significant  effect  was  observed  with 
medication  errors.  This  1  hour  decrease  in  nursing  care  hours  was  also  associated  with 
a  43  and  52%  increase  in  needlestick  injuries  on  medical  surgical  and  critical  care  units, 
respectively. 

The  percentage  of  staff  on  a  shift  who  were  DoD  civilians  had  several  interesting 
associations.  A  10%  decrease  in  the  percentage  of  civilian  staff  on  a  given  shift  was 
associated  with  a  33-48%  increased  likelihood  of  falls,  as  much  as  a  67%  increased 
chance  of  MAEs,  as  high  as  a  54%  increase  in  needlestick  injuries  and  decrease  in 
hospital  acquired  pressure  ulcer  prevalence.  To  help  explain  this  relationship,  we 
examined  the  differences  in  demographics  between  military  and  civilian  nurses  from  a 
cross-sectional  annual  survey  of  nurses  from  our  participating  hospitals.  Noteworthy 
among  the  differences  was  the  level  of  experience.  Military  nurses  had  on  average  9 
years  less  experience  than  the  civilian  nursing  personnel  (5  versus  14  years, 
respectively;  t  =  -17.88,  p  <  0.001).  A  similar  difference  was  also  observed  in  a 
separate  study  using  a  different  military  hospital  sample  (Patrician,  Shang,  &  Lake, 
2010).  This  experience  level  difference  reflects  a  distinction  in  career  expectations 
between  military  and  civilian  nurses.  Military  nurses  typically  begin  their  careers  as 
direct  care  provider  staff  nurses  and  are  expected  to  advance  into  leadership  positions 
and  away  from  direct  care.  Civilian  nurses  do  not  have  this  expectation,  infrequently 
hold  non-direct-care  positions,  and  are  usually  hired  with  at  least  1  year  of  experience. 
Our  study’s  personnel  category  may  be  serving  as  a  proxy  for  experience,  which  other 
researchers  have  found  to  be  associated  inversely  with  MAEs  and  falls  (Blegen, 
Vaughan,  &  Goode,  2001). 

Our  needlestick  findings  with  relationship  to  experience  are  similar  to  those  of  Clark 
(2007)  who  examined  surveys  from  1 1,516  Pennsylvania  nurses  and  found  that  nurses 
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with  less  than  5  years  of  experience  were  more  likely  to  have  sustained  a  needlestick 
injury  in  the  preceding  year.  Experienced  nurses  have  likely  found  ways  to  manage 
work  demands  more  efficiently,  not  only  to  protect  themselves  from  needlestick  injuries 
but  also  to  have  a  protective  effect  on  the  other  staff  working  with  them  on  a  shift. 
Therefore,  administrators  must  consider  the  level  of  experience,  and  not  simply  total 
numbers  of  staff,  when  scheduling  their  workforce. 

In  analyzing  shift  times,  41%  fewer  MAEs  and  45%  fewer  needlestick  injuries  occurred 
on  critical  care  units  at  night.  This  lower  incidence  may  have  resulted  because  fewer 
medications  are  administered  during  night  shifts,  fewer  new  or  modified  medication 
orders  are  received  on  night  shifts  and  therefore  fewer  opportunities  for  error  would 
exist,  or  there  may  be  fewer  interruptions  during  night  shift  medication  dispensing. 
Likewise,  there  may  be  fewer  opportunities  for  needlesticks  not  only  because  of  fewer 
medications  administered,  but  because  fewer  invasive  procedures  are  done  at  night. 

Although  we  found  a  decrease  in  falls,  medication  errors,  and  pressure  ulcers  over  the 
duration  of  the  study  (Loan,  Patrician,  &  McCarthy,  2011),  there  was  no  such  trend  in 
restraint  prevalence.  However,  we  did  find  small  but  significant  increases  in  restraint 
use  with  a  higher  proportion  of  contract  nurses  in  critical  care  and  a  3.89%  lower 
prevalence  of  restraint  use  with  each  additional  hour  of  RN  staff  per  patient  per  shift. 

The  prevalence  of  hospital  acquired  pressure  ulcers,  Stage  2  (HAPU2)  were  higher  in 
critical  care  units,  despite  the  finding  that  average  Braden  scores  were  fairly  similar 
across  all  unit  types.  In  critical  care,  however,  each  1  point  increase  in  the  Braden 
score  was  associated  with  an  1 1 .2%  increase  in  HAPU2  rates.  When  classifying  units 
as  either  good  or  poor  performers  relative  to  Braden  scale  and  pressure  ulcer  rates, 
good  performers  had  a  higher  contract  mix  and  a  higher  number  of  patients  per  RN  as 
compared  to  poor  performers.  These  findings  defy  explanation  and  we  will  address  our 
staffing  measure  used  for  the  pressure  ulcer  analysis  under  the  limitations  section  of 
this  report. 

Patient  satisfaction  with  various  aspects  of  care  was  high  across  all  hospitals.  Our  only 
explanation  is  that  once  patients  enter  a  military  hospital,  the  care  they  receive  meets 
their  expectations,  but  we  did  not  explicitly  test  this.  Of  note,  the  Veterans  Health 
Administration  (VA)  also  reports  unusually  high  inpatient  satisfaction  (Oliver,  2007)  It 
may  be  that  federally  administered  and  provided  health  care  is  actually  superior  to 
civilian  healthcare  because  of  higher  quality,  less  budgetary  (and  care)  restrictions, 
increased  public  scrutiny,  or  may  be  reflective  of  characteristics  of  those  who  choose  to 
work  in  military  or  VA  hospitals. 

Nurse  ratings  of  their  practice  environment  was  more  favorable  in  military  nurses, 
perhaps  because  military  staff,  more  so  than  civilian  staff,  are  expected  to  participate  in 
hospital  affairs,  in  continuing  education  and  professional  development,  and  to  be 
groomed  to  take  on  leadership  roles.  An  increase  in  total  nursing  care  hours  was 
associated  with  a  more  favorable  rating  of  the  practice  environment  overall,  perhaps 
because  better  work  environments  tend  to  have  better  staffing.  This  was  observed 
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most  notable  in  the  Staffing  and  Resource  Adequacy  (SRA)  subscale,  where  we 
document  a  statistically  significant  association  with  actual  total  nursing  care  hours. 

Being  an  RN  was  associated  inversely  with  the  SRA  subscale,  perhaps  because  RNs 
feel  the  brunt  of  poor  staffing  more  so  than  do  LPNs  or  other  staff,  as  they  must 
supervise  all  staff  and  oversee  care  for  the  unit. 

Nurse  job  satisfaction,  contrary  to  many  anecdotal  reports  in  the  literature,  was  not 
found  to  be  associated  with  actual  staffing  nor  RN  skill  mix.  This  was  a  surprising  and 
unexpected  finding,  and  the  first  time  actual  staffing  was  used  as  an  independent 
variable  to  predict  job  satisfaction,  leading  us  to  the  conclusion  that  it  is  not  poor  staffing 
that  causes  job  dissatisfaction,  but  other  conditions  such  as  work  environment,  but  we 
did  not  test  this  association.  Satisfaction  was  dependent  more  so  on  ones'  position  in 
the  hierarchy  of  the  unit  than  on  any  of  the  staffing  or  patient  turnover  rates.  Military 
LPNs  and  unlicensed  personnel  were  the  least  satisfied  of  the  categories  and  military  in 
general  tended  to  be  less  satisfied  than  civilians.  RNs  were  more  satisfied  than  non- 
RNs.  This  may  be  explained  by  the  additional  duties  the  military  are  required  to 
perform,  such  as  covering  when  civilians  call  in  sick.  Theoretically  military  staff  are 
available  24  hours  a  day,  7  days  a  week  and  must  abide  by  orders  from  superiors. 


Conclusions  and  Implications 

The  use  of  multiple  data  collection  sites  permitted  the  accumulation  of  an 
extraordinarily  large  number  of  data  points  that  may  allow  the  investigators  to  uncover 
relationships  among  the  structural  and  outcome  variables.  The  use  of  13  sites  permitted 
the  investigators  to  use  hospital  size  as  a  control  variable  and  unit  type  as  a  variable  of 
interest.  This  large  dataset  has  rich  variability  which  may  allow  for  discovery  of  new 
relationships.  The  limitations  of  using  multiple  data  collection  sites  is  the  systematic 
error  that  can  be  introduced  and  the  confounding  effects  of  unmeasured  but  present 
hospital  and/or  unit-level  variables.  Using  hospital  size,  unit -type  and  other  site  specific 
factors  as  control  variables  may  minimize  their  confounding  effect  on  the  analysis. 


Several  very  important  questions  remain,  although  their  analyses  was  beyond  the  scope 
of  this  project.  They  are  as  follows: 

1 .  How  does  patient  turbulence  affect  the  adverse  events  in  this  study?  This  can  be 
done  by  simply  using  the  data  that  is  available  or,  more  correctly,  using  an  acceptable 
imputation  method.  The  latter  would  require  several  sequential  steps.  First,  we  would 
need  to  determine  the  missing  data  mechanism  in  the  ADT  variable  (missing  at  random, 
missing  completely  at  random,  or  non-random  missingness).  Based  upon  the  results, 
determine  an  imputation  strategy  and  conduct  the  analysis  as  originally  proposed.  As  a 
test  of  the  validity  of  the  imputation  models,  test  the  results  obtained  against  simulated 
missing  data  of  the  same  mechanism  by  using  the  portion  of  the  data  set  with  no 
missing  values,  removing  a  random  percentage  of  values,  and  imputing  using  the  same 
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models.  This  analysis  will  not  only  answer  the  research  question  #1  as  originally  posed, 
but  will  add  to  the  methodological  knowledge  on  imputation  methods  for  various  types  of 
missing  data  mechanisms. 

2.  What  are  the  effects  of  the  practice  environment  on  the  relationships  between  the 
predictor  and  outcome  variables  in  the  shift  level  analysis.  This  is  a  complicated 
analysis  because  of  the  two  possible  and  distinct  relationships  of  the  practice 
environment  (and  separate  components  of  it)  on  the  predictor-outcomes  relationship. 
The  environment  may  act  as  a  moderator  of  the  relationship,  affecting  it  differently  as 
the  practice  environment  is  better  or  is  worse.  As  an  example,  a  good  practice 
environment  may  lessen  the  effect  of  poor  staffing  on  outcomes  because  in  a  positive 
practice  environment,  workers  could  better  mobilize  resources  such  as  team  work  and 
cooperation  with  other  departments  to  get  the  work  done  safely.  On  the  other  hand, 
unfavorable  work  environments  may  worsen  the  effects  of  low  staffing  on  outcomes 
such  that  poor  outcomes  would  be  experienced  at  relatively  higher  staffing  than  would 
be  expected.  There  is  some  empirical  evidence  to  suggest  that  this  is  the  case; 
however  it  has  never  been  tested  at  the  shift  level. 

Alternatively,  there  could  be  a  mediating  effect  of  the  practice  environment  on  outcomes 
such  that  better  staffing  helps  to  create  a  positive  work  environment,  which  in  turn, 
affects  the  outcomes  in  a  positive  way. 

3.  What  effect  does  the  practice  environment  and  its  subcomponents  have  on  nurse  job 
satisfaction.  This  is  an  important  question  and  one  that  was  not  posed  in  the  original 
study.  There  is  a  growing  body  of  evidence  suggesting  that  it  is  the  work  environment 
and  not  necessarily  staffing  that  satisfies  or  dissatisfies  nurses.  Our  analysis  did  not 
show  any  effect  of  staffing  on  nurse  job  satisfaction. 

4.  Should  the  PES  should  be  used  for  all  skill  levels  of  nursing  personnel?  We  do  not 
know,  for  example,  whether  unlicensed  assistive  personnel  and  registered  nurses  would 
rate  the  same  unit  environment  in  the  same  manner. 

5.  What  are  the  patient  level  risk  factors  for  pressure  ulcer  development?  A  recent 
publication  by  Chapman  and  Kane  (2010)  indicated  an  increased  risk  of  pressure  ulcers 
among  war  wounded  service  members.  Our  data  could  help  explore  that  association  in 
more  detail  given  that  we  have  collected  physiologic  data  (lab  values)  known  to  be  risk 
factors  for  pressure  ulcer  development. 

6.  If  we  used  a  different  measure  of  staffing  in  the  pressure  ulcer  analysis,  would  our 
results  differ?  In  looking  back  at  the  staffing  metric  used  (staffing  on  the  day  of  the 
pressure  ulcer  prevalence  study),  several  anecdotal  factors  came  to  mind.  First, 
additional  staff  were  scheduled  on  the  day  of  the  pressure  ulcer  prevalence  studies  in 
all  facilities,  because  the  research  team  had  requested  staff  nurse  participation  in  the 
study.  Those  excess  staff  would  have  been  counted  within  their  respective  shifts  had 
they  gone  back  to  work  the  remainder  of  the  shift  following  the  pressure  ulcer  survey, 
thus,  staffing  may  have  been  over-inflated  on  the  day  of  the  pressure  ulcer  prevalence 
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study.  The  research  team  now  recommends  looking  at  staffing  48  hours  prior  to  a 
pressure  ulcer  survey  -  this  measure  would  fit  within  the  physiologic  time  of  the 
development  of  a  pressure  ulcer. 


Limitations 

Several  limitations  must  be  mentioned.  First,  this  study  had  two  phases  in  that  it  was 
both  database  building  and  database  analysis,  and  therefore,  we  did  not  know  a  priori 
which  particular  variables  would  be  in  the  final  analysis,  because  of  reliability,  validity, 
and  missing  data  concerns.  The  amount  of  missing  data  precluded  some  of  the 
analysis.  Second,  no  causation  could  be  implied  since  this  was  an  exploratory,  and  not 
an  experimental  design. 

Third,  the  use  of  self-reported  unusual  occurrence  reports  is  frequently  cited  as  a 
limitation.  Many  claim  that  reporting  is  a  function  of  the  patient  safety  climate,  and  not  a 
true  indication  of  the  frequency  of  adverse  events.  We  accept  that  we  likely  have 
underreporting  of  adverse  events;  however  the  fact  that  we  find  associations  in  light  of 
both  under-reporting  and  rare  events  is  noteworthy.  Many  researchers  suggest  that 
falls  are  the  most  frequently  reported  of  adverse  events  because  they  cannot  be  hidden 
from  other  staff  members  as  perhaps  medication  errors  can.  In  our  data  set,  we  find 
more  medication  errors  reported  that  we  do  falls.  Needlestick  injuries  have  been  shown 
to  be  underreported  when  using  actual  injury  reports  as  compared  to  retrospective  self 
reports  (Aiken,  Sloane,  &  Klocinski,  1997);  however,  retrospective  reports  are  also  error 
prone  due  to  recall  bias.  The  best  way  to  document  adverse  events  is  through 
observation,  but  observation  is  extremely  expensive  making  it  unrealistic  for  a  study  that 
spans  four  years  and  thousands  of  shifts. 

A  fourth  limitation  was  the  selection  of  staffing  measure  to  use  in  our  pressure  ulcer 
prevalence  analysis.  In  hindsight,  we  discovered  that  units  were  typically  staffed  at 
higher  levels  during  the  prevalence  study  days  because  of  the  need  to  provide  staff 
nurses  for  part  of  the  day  to  conduct  the  surveys.  Although  their  time  away  from  the 
unit  to  conduct  the  surveys  should  not  have  been  counted  against  the  staffing  hours  for 
that  day,  extra  staff  were  on  hand  on  the  units  to  assist  the  research  teams  with  tasks 
such  as  turning  the  patients. 

In  contemplating  the  appropriate  staffing  metric  to  use  for  the  pressure  ulcer  -  staffing 
analyses,  it  became  apparent  that  staff  shortages  on  different  days  are  theoretically 
associated  with  the  different  stage  of  development  of  the  ulcer.  For  example,  to 
determine  if  staffing  was  a  factor  in  the  development  of  the  Stage  1  pressure  ulcers 
detected  on  the  pressure  ulcer  prevalence  study  day,  we  would  need  to  look  back  to 
staffing  in  the  24  hours  prior  to  the  survey.  Likewise,  to  detect  staffing  effects  on  HAPU 
Stage  Its  or  higher,  we  would  need  to  look  at  staffing  backwards  in  time  48-72  hours, 
because  of  the  time  factors  in  the  physiologic  development  of  Stage  II  HAPUs.  We 
have  the  data  to  conduct  these  analyses  to  determine  if  there  is  a  staffing  effect  at 
different  times  prior  to  the  survey.  Knowing  which  staffing  measure  to  use  would  inform 
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future  research  in  this  area.  This  is  yet  another  contribution  to  the  field  that  can  be 
answered  with  MilNOD  data. 

A  final  limitation  was  not  collecting  specific  data  on  interventions  that  were  being 
conducted  at  the  time  of  the  MilNOD.  This  was  of  course,  beyond  the  scope  of  the 
study.  However,  having  these  data  could  help  us  to  understand  the  changes  in  adverse 
events  over  time  that  our  data  show.  We  can  only  say  that  there  was  a  change,  not  that 
the  MilNOD  or  anything  else  for  that  matter,  helped  produce  the  change.  This  leads 
back  to  the  second  limitation,  the  inability  to  attribute  causation,  even  on  a  small  scale  in 
isolated  units  that  may  have  had  performance  improvement  projects  to  address  some  of 
the  adverse  events. 


Significance  of  Research  to  Military  Nursing 

•  The  findings  from  this  study  support  a  working  model  for  collecting  and 
disseminating  reliable,  valid,  and  usable  data  across  hospitals  to  support  patient 
safety. 

•  Promoting  the  use  of  high-quality  inpatient  data  can  contribute  to  the 
development  of  evidence-based  policies  and  procedures  to  monitor  the  effect  of 
nurse  staffing  on  clinical  and  service  outcomes. 

•  The  MilNOD  was  a  first  attempt  in  the  military  to  collect  performance  data  to 
compare  like  units  in  like  hospitals  with  each  other  on  indicators  important  to 
nursing. 

•  Although  the  military  can  and  does  adopt  civilian  research  findings  to  its 
management  practices,  civilian  data  is  limited  in  that  the  military  has  different 
categories  of  providers;  we  have  shown  here  that  this  does  make  a  differences  in 
outcomes. 

•  Each  shift  must  be  staffed  adequately  not  only  with  numbers  of  staff,  and  skill 
levels,  but  also  with  the  right  mix  of  experienced  (i.e.,  civilian)  staff. 

•  These  data  are  useful  for  military  nursing  leaders  to  defend  existing  staffing 
levels  and/or  to  justify  additional  staffing  needs  to  their  respective  commanders. 

•  These  data  are  useful  for  leaders  to  justify  budgetary  requirements  for  staffing. 

•  An  adaptation  of  this  model  is  currently  being  used  as  an  interdisciplinary  clinical 
outcomes  database  for  patient  safety  metrics  in  Army  medical  facilities 
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improve  patient  &  cost  outcomes.  2007  Military  Health  System  Conference, 
Washington,  DC. 
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Lay  Press 

The  MilNOD  Messenger  was  distributed  quarterly  as  a  means  of  keeping  all  study  team 
members  connected  to  grant  activities. 


Possible  Policy  Implications 

The  MilNOD  team  was  approached  by  the  Army  Medical  Command  Patient  Safety 
Office  to  assist  with  a  service-wide  initiative  to  incorporate  nursing-sensitive  outcomes 
into  a  large  patient  safety  database.  It  is  anticipated  that  several  MilNOD  structural 
variables  (skill  mix,  nursing  care  hours  per  patient  day,  etc.)  will  be  included  in  this 
initiative. 


Possible  Change  of  Practice 

Nurse  Managers  at  various  MilNOD  sites  have  used  their  MilNOD  data  to  re-examine 
staffing  ratios  and  to  convert  paraprofessional  (e.g.  LPN)  staff  to  professional  (RN)  staff 
positions  in  order  to  enhance  quality  nursing  care.  One  site  in  particular,  assigned  more 
new  military  staff  to  the  medical-surgical  units  with  the  lowest  RN  skill  mix. 

Many  sites  have  used  the  outcome  data  for  falls,  medication  administration  errors,  and 
pressure  ulcer  prevalence  to  focus  on  strategies  to  decrease  incident  rates  and  improve 
patient  safety.  An  example  of  this  would  be  the  medical  and  stepdown  units  of  one  large 
medical  center  who  both  decided  to  launch  a  patient  safety  initiative  to  decrease  fall 
rates  using  targeted  strategies  such  as  consistent  documentation  of  the  Fall  Risk  score, 
bed  rounds  to  ensure  alarms  were  activated  and  beds  were  in  the  low  position,  high  risk 
patients  were  identified  and  moved  closer  to  the  nursing  station,  high  risk  patients  were 
checked  and  offered  assistance  every  2  hours,  devices  such  as  floor  mat  alarms  were 


66 


Col  (ret)  Patrician  A.  Patrician 


Proposal  No  N03-P07 


purchased,  and  a  white  board  at  the  nursing  station  tracked  how  many  fall-free  days 
had  occurred.  These  measures  have  been  adhered  to  for  over  a  year  and  continue  to 
be  key  aspects  in  the  success  of  the  patient  safety  initiative  to  reduce  falls. 

Another  example  of  a  change  in  practice  is  related  to  the  pressure  ulcer  prevalence 
surveys  conducted  as  part  of  the  MilNOD  Study.  Several  facilities  had  never  done 
pressure  ulcer  prevalence  surveys  before  participating  in  MilNOD.  With  the  assistance 
of  several  core  MilNOD  team  members  the  facilities  were  able  to  educate  nurses  about 
skin  care  and  staging  of  pressure  ulcers,  organize  a  team  to  inspect  the  skin  of  all 
patients  in  the  hospital  on  a  given  day,  and  identify  areas  needing  improvement  related 
to  pressure  ulcer  prevention.  Training  materials,  including  photographs  to  practice 
staging  of  pressure  ulcers,  were  made  available  to  any  facility  upon  request.  Electronic 
consultations  were  also  conducted  with  remote  sites  when  they  faced  challenging 
wound  care  issues 

Table  23  provides  information  solicited  from  MilNOD  sites  regarding  their  prevention 
activities. 

Table  23 


MilNOD  Pressure  Ulcer  Practice  Enhancements  across  Participating  MTFs  (As  of  Feb 
06) 


%  Of  Participating  Sites  .  .  . _ 

>  Collecting  Regular  Hospital  Acquired 
Pressure  Ulcer  Data 


Before  MilNOD 
36% 


>  Using  An  Evidence-Based  Pressure  Ulcer 
Risk  Assessment  Tool 


After  MilNOD 
100% 

100% 


>  Conducting  A  Pressure  Ulcer  Prevalence  36%  100% 

Study  At  Least  Yearly 

>  Using  Pressure  Ulcer  Staging  Criteria  25%  100% 

That  Are  Evidence-Based  &  Current 
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Appendix  B 

Problems  Encountered  and  Resolutions 


The  problem  causing  the  longest  delay  to  this  study  was  the  IRB  approval  process  at 
participating  sites.  Each  IRB  requested  different  materials  to  be  forwarded  for  review 
and  revisions  to  the  proposal  were  often  required  prior  to  final  approval.  One  small  issue 
delayed  this  approval  for  several  months;  the  impact  depended  on  the  timelines  of  the 
IRB  reviewing  schedule.  Additionally,  Operation  Iraqi  Freedom  (OIF)  deployments  were 
unanticipated  and  affected  study  implementation  and  progression,  impacting  personnel 
in  all  of  the  study  facilities  and  affecting  multiple  facets  of  the  study  including  the  IRB 
process,  site  coordinators,  and  all  levels  of  nursing  leaders/managers.  Deployment- 
related  setbacks  resulted  in  the  need  for  additional  training  and  briefings  to  site 
personnel  and  multiple,  repeat  explanations  to  Head  Nurses  and  Chief  Nurses. 

A  current  DoD  Assurance  of  Compliance  with  Human  Subjects  Protections  is  required 
for  each  DoD  institution  conducting  human  subjects  research.  Review  of  the  MilNOD 
protocol  for  the  Oak  Harbor  Naval  Hospital  was  delayed  until  a  new  DoD  Assurance  of 
Compliance  could  be  approved  (approximately  12  months)  by  the  Navy  Bureau  of 
Medicine  and  Surgery.  The  DoD  Assurance  of  Compliance  with  Human  Subjects 
Protections  was  finally  obtained  for  Oak  Harbor  Naval  Hospital  Winter  2004.  This 
unanticipated  requirement  provided  an  opportunity  for  the  research  team  to  learn  the 
extent  of  preparatory  work  that  may  be  required  when  other  small  MTFs  are  brought 
into  the  MilNOD  study. 

Eventually  IRB  approval  was  obtained  at  all  of  the  7  sites,  representing  three  branches 
of  the  military,  Army,  Navy,  and  Air  Force.  Joint  IRB  approval  for  the  two  Navy  facilities, 
Oak  Harbor  Naval  Hospital  and  Naval  Hospital  Bremerton  came  from  the  regional  IRB 
Headquarters  at  Naval  Medical  Center  San  Diego  when  a  protocol  was  filed  for  MilNOD 
IV.  However,  approval  was  not  received  in  time  for  the  two  original  Navy  sites  to 
participate  in  MilNOD  III.  To  highlight  how  time-consuming  the  problem  was  with  the 
IRB  process,  the  initial  protocol  packet  was  submitted  in  January  2003  and  approval 
was  only  granted  in  December  2004.  The  study  progresses  as  much  as  possible  while 
waiting  for  the  final  approval.  Projected  personnel  for  all  sites  have  been  hired  or 
assigned  to  the  team  and  trained  for  the  study.  The  MilNOD  study  team  conducted  site 
visits  with  Chief  Nurses  and  reviewed  study  aims  and  site  coordinator  responsibilities  at 
all  seven  original  sites.  Due  to  the  delay  in  IRB  approval  for  the  Navy  sites  we  were 
unable  to  fully  meet  our  first  aim.  The  Navy  sites  began  data  collection  in  September 
2005. 

A  variety  of  other  problems  were  encountered  including  identification  of  site-specific 
data  sources,  designation  of  data  collectors,  data  acquisition  and  data  management. 
Onsite  training  issues  related  to  the  database  were  as  expected  due  to  the  high  level  of 
security  required  for  the  data  collected  in  this  project  and  the  lack  of  familiarity  with 
highly  technological  systems  by  most  study  personnel.  Initially  a  File  Transfer  Point 
(FTP)  was  set  up  on  the  MAMC  server  which  worked  well  until  the  MAMC  Information 
Management  Office  directed  the  termination  of  the  FTP  facility-wide  due  to  a  new 
requirement  for  enhanced  security.  This  resulted  in  the  creation  of  the  Microsoft  product 
called  a  SharePoint  portal  that  allows  collaboration  between  multiple  sites  for  the 
purpose  of  sharing  documents  and  data  with  associated  Windows  securities.  This  is  a 
highly  secure  password-protected  site  that  is  considered  the  preferred  tool  for 


Government  communications  and  data  sharing.  All  site  personnel  received  instructions 
about  using  this  portal  and  were  frequently  taken  through  the  site  step-by-step  by 
MAMC  study  staff.  We  have  access  to  the  MAMC  developer  by  phone  and  email  7 
days  a  week  to  assist  with  troubleshooting.  It  is  an  individual’s  responsibility  to  keep 
their  password  current  to  access  the  portal  and  instructions  and  reminders  are  sent  out 
frequently.  Occasionally  the  server  is  inaccessible  for  short  periods  of  time  but  we  know 
of  no  recent  or  chronic  issues  with  the  SharePoint  portal. 

A  final  problem  was  statistical  support.  Because  the  statistician  who  originally  agreed  to 
perform  the  analyses  for  this  project  was  ultimately  unable  to  assist,  the  CALNOC 
Consultants  were  approached  in  September  2007  and  they  agreed  to  coordinate  the 
analysis  of  seven  of  the  analysis  specific  aims;  completing  the  entire  analysis  was 
beyond  the  scope  of  the  remaining  MilNOD  budget  at  that  time.  Dr.  Moshe  Fridman 
performed  the  analysis  and  on  August  10-1 1  of  2008,  COL(ret)  Patrician,  Dr.  Loan,  and 
Dr.  McCarthy  traveled  to  the  University  of  San  Francisco  to  meet  with  team  consultants 
Dr.  Nancy  Donaldson  and  Dr.  Fridman.  The  one-day  visit  was  spent  reviewing  the  data 
preparation,  model  configurations,  and  statistical  tests  performed.  Subsequently  all 
study  findings  to  date  were  discussed  with  full  explanations  by  Dr.  Fridman  of  his 
analytical  methods.  He  also  provided  expert  insight  and  interpretation  of  the  data.  Dr. 
Fridman  analyzed  most  of  the  aims;  the  remainder  were  analyzed  by  Dr.  Andres 
Azuero,  funded  by  Dr.  Patrician's  endowment. 
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Appendix  C 

Psychometric  Report  #1 


Reliability  and  Validity  of  Measures:  PSNQ 


If  no  instrumentation  was  used  for  your  study,  check  here  I  I 


Directions:  Please  complete  the  questions  below  addressing  demographic  characteristics  of  your 
sample  and  overall  sample  size.  For  the  tool  identified  in  the  attached  cover  letter,  please 
complete  the  following  questions  regarding  any  reliability  and/or  validity  testing  you  performed. 
Please  note  that  this  list  is  not  meant  to  be  exhaustive.  If  you  performed  other  reliability  and/or 
validity  testing  which  is  not  listed,  please  identify  the  test,  and  report  your  findings  under  “other.”  If 
further  space  is  needed,  please  attach  additional  pages.  Please  submit  a  copy  of  the  tool  if  you 
made  any  modifications. 


Principal  Investigator- Contact  Information 


Name: 


Address: 


Title  of 
Study 


COL  (ret)  Patrician  A.  Patrician 


Telepho 

ne 


Number: 


E-mail: 


Work 


Home 


Military  Nursing  Outcomes  Database  (MilNOD  IV):  Analysis  and  Expansion 


Demographic  Characteristics  of  Sample 


Total  sample  size: 

Age 

Range: 

N/A 

Number 

Service 

<19  yrs 

19-60 

yrs 

>60  yrs  Other 

See 

description 

below 

Army 

Male 

Not  assessed 

Air  Force 

Female 

Navy 

Marine 

Number 

Race: 

Number 

Service  Component: 

Not 

assessed 

Caucasian 

Not 

assessed 

Active  Duty 

African-American 

Retired 

Hispanic 

Reserve 

Asian/Pacific  Islander 

National  Guard 

Other  (Describe) 

Dependent 

Briefly  describe  defining  characteristics  of  sample: 
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Total  sample  size:  5  MTFs;  26  nursing  units;  1170  nursing  personnel  and  1195  patients. 

Service  not  specifically  assessed,  however,  there  was  one  Air  Force  MTF  and  3  Army  MTFs  in 
the  sample. 


Instrument  Reference 

Instrument  Patient  Satisfaction  with  Nursing  Care  Questionnaire 

Number  » 

°f  A  O 

0  .  used  2 

Scales: 

Title:  (PSNCQ) 

Instrument 

Publication  1997 

Year: 

Edition:  NA 

Authors:  Jacox,  Bausell,  &  Marenholz 

Publisher: 

TKIe"-03'  °r  Outcomes  Management  for  Nursing  Practice 

Year:  1997  Volume:  1 

Page  Numbers:  20-28 

Tool  Modifications 

Did  you  modify  this  M  Yes  (Answer  A  &  B  below)  □  No 

tool? 

A.  Briefly  describe  The  instrument  did  not  have  a  question  on  global  satisfaction  with 

why  modifications  nursing  care,  which  was  an  indicator  the  research  team  wished  to 

were  made:  capture. 

B.  Describe  what 

modifications  were  Added  one  question  on  overall  satisfaction  with  nursing  care. 

made: 

Directions:  Please  indicate  any  reliability  and/or  validity  testing  you  did  on  this  instrument. 

Please  report  findings  of  each  scale  next  to  the  test. 

Check  al 

that  apply 

Reliability 

Validity 

□  Internal-Consistency  Reliability 

Content  Validity 

HI  Cronbach  Coefficient  Alpha  .98  for 

instrument 

□  Index  of  Content  Validity  (CVI) 

□  Kuder-  Richardson  (KR-20) 

□Other  (please  describe  on  back  of  form) 

□  Interrator  Reliability 

Criterion-Validity 

□  Intrarater  Reliability 

□  Predictive 

□  Coefficient  of  Stability  (test-retest) 

□  Linear  Correlation 

□  Coefficient  of  Equivalence 

Name  of  Criterion  Measure  Used: 

□  Other  (please  describe  on  back  of  form) 

□  Concurrent 

□Linear  Correlation 
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Reliability  of  Individual  Scales 

Name  of  Criterion  Measure  Used: 

Scale  Name 

Reliability 

□  Construct  Validity  (include  a  copy  of  findings) 

Patient  Satisfaction  with: 

□  Multitrait-Multimethod 

Patient  Education 

.97 

□  Hypothesis  testing 

Discharge  Teaching 

.99 

□  Contrasted  Group 

□  Factor  Analysis 

□  Exploratory 

□  Confirmatory 

Please  use  back  of  form  for  additional  scales 

□  Other  (please  describe  on  back  of  from) 

Evaluation  of  Measure 

Would  you  recommend  the  use  of  this  measure  in  your  population  to  other  researchers? 

HI  Yes.  Please  explain 
why. 

This  is  a  good  instrument  to  measure  satisfaction  with  hospital 
experience  and  more  specifically  nursing  care.  The  instrument  has 
four  single  item  measures:  satisfaction  with  hospital,  food,  medical 
care,  and  nursing  care.  It  also  has  three  subscales  (of  which  we 
used  one):  Satisfaction  with  nurse’s  technical  ability,  caring,  and 
teaching.  All  three  have  high  Cronbach  alphas:  .95,  .98,  and  .97 
respectively.  Because  of  our  specific  indicators,  we  were  able  to 
pick  items  out  of  this  instrument  to  use  as  single  item  measures 
(satisfaction  with  pain  management  and  overall  nursing  care).  In  this 
regard,  the  instrument  is  quite  flexible. 

□  No  Please  explain  why. 
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Psychometric  Report  #2 


Reliability  and  Validity  of  Measures:  PES 


If  no  instrumentation  was  used  for  your  study,  check  here  I  I 


Directions:  Please  complete  the  questions  below  addressing  demographic  characteristics  of  your 
sample  and  overall  sample  size.  For  the  tool  identified  in  the  attached  cover  letter,  please 
complete  the  following  questions  regarding  any  reliability  and/or  validity  testing  you  performed. 
Please  note  that  this  list  is  not  meant  to  be  exhaustive.  If  you  performed  other  reliability  and/or 
validity  testing  which  is  not  listed,  please  identify  the  test,  and  report  your  findings  under  “other.”  If 
further  space  is  needed,  please  attach  additional  pages.  Please  submit  a  copy  of  the  tool  if  you 
made  any  modifications. 


Principal  Investigator- Contact  Information 


Name: 


COL  (ret)  Patrician  A.  Patrician 


NB  324,  1530  3^Avenue  South 


Telepho 

ne 


Work 


Address: 


Number: 


Home 


Birmingham,  AL  35294-1210 


E-mail: 


Ppatrici@uab.edu 


Title  of 
Study 


Military  Nursing  Outcomes  Database  (MilNOD  IV):  Analysis  and  Expansion 


Demographic  Characteristics  of  Sample 


Total  sample  size: 

Age 

Range: 

See  description 
below 

Number 

Service 

<19  yrs 

19-60 

yrs 

>60  yrs  Other 

X 

Army 

Male 

Gender  not  assessed 

X 

Air  Force 

Female 

Navy 

Marine 

Number 

Race: 

Number 

Service  Component: 

Race  not 
assessed 

Caucasian 

X 

Active  Duty 

African-American 

Retired 

Hispanic 

X 

Reserve 

Asian/Pacific  Islander 

National  Guard 

Other  (Describe) 

Dependent 

Briefly  describe  defining  characteris 


ics  of  sample: 


Total  sample  size 


Age  ranges: 


2003:  5  MTFs,  228  RNs,  123  LPNs  and  aides  (351  total) 
2004:  7  MTFs,  303  RNs,  177  LPNs  and  aides  (480  total) 

2003:  19-66  years  (RN  22-64  yrs;  LPN/aides  19-66  yrs) 
2004:  19-66  years  (RN  22-66  yrs;  LPN/aides  19-66  yrs) 
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Instrument  Reference 


Instrument  Title: 


Practice  Environment  Scale  of  the  Nursing  Number  5  +  total 

Work  Index  (PES-NWI) of  Scales:  score 


Instrument 
Publication  Year: 


2002 


Edition: 


NA 


Authors: 


Lake,  Eileen 


Publisher: 


Wiley  Periodicals 


Journal  or  Book 
Title: 


Research  in  Nursing  &  Health 


Year: 


2002  Volume:  25 


Page  Numbers: 


176-188 


Tool  Modifications 


Did  you  modify  this  tool? 


□  Yes  (Answer  A  &  B  below)  0  No 


A.  Briefly  describe  why 
modifications  were  made: 


We  used  the  NWI-R  as  reported  in  Aiken  &  Patrician  (2000), 
but  for  the  analysis,  we  used  the  PES-NWI  items  from  the 
NWI-R. 


B.  Describe  what 

modifications  were  made: 


Directions:  Please  indicate  any  reliability  and/or  validity  testing  you  did  on  this  instrument.  Please  report  findings  of 
each  scale  next  to  the  test. 


Check  all  that  apply 


Reliability 

Validity 

□  Internal-Consistency  Reliability 

Content  Validity 

0  Cronbach  Coefficient  Alpha  See  Below 

□  Index  of  Content  Validity  (CVI) 

□  Kuder-  Richardson  (KR-20) 

□Other  (please  describe  on  back  of  form) 

□  Interrator  Reliability 

Criterion-Validity 

□  Intrarater  Reliability 

□  Predictive 

□  Coefficient  of  Stability  (test-retest) 

□  Linear  Correlation 

□  Coefficient  of  Equivalence 

Name  of  Criterion  Measure  Used: 

□  Other  (please  describe  on  back  of  form) 

□  Concurrent 

□Linear  Correlation 

Reliability  of  Individual  Scales 

Name  of  Criterion  Measure  Used: 

Scale  Name 

Items 

Reliability 

□  Construct  Validity  (include  a  copy  of  findings) 

2003 

2004 

PES-NWI 

29 

.92 

.93 

□  Multitrait-Multimethod 

Nsg  Participation  in  Hosp 

Affairs 

9 

.84 

.85 

□  Hypothesis  testing 

Nsg  Foundations  for  Qual  Care 

9 

.79 

.82 

□  Contrasted  Group 

Nsg  Manager’s  Ability,  Ldrshp 

4 

.81 

.83 

0  Factor  Analysis 

Staffing  &  Resource  Adequacy 

4 

.77 

.79 

0  Exploratory  See  explanation  below 

Nurse-MD  Collaboration 

3 

.83 

.77 

□  Confirmatory 

Please  use  back  of  form  for  additional  scales 

□  Other  (please  describe  on  back  of  from) 

Evaluation  of  Measure 


Would  you  recommend  the  use  of  this  measure  in  your  population  to  other  researchers? 

0  Yes.  Please  explain  I  would  recommend  using  the  PES-NWI,  but  not  the  NWI-R  in  its 
why. _ entirety  due  to  respondent  burden. _ 

□  No  Please  explain  why. 
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Exploratory  Factor 
Analysis: 


An  exploratory  factor  analysis  was  conducted  on  the  2003  and  2004 
data  sets  using  principal  components  analysis.  In  the  2003  data  set, 
7  factors  with  Eigenvalues  greater  than  1  were  extracted  and 
accounted  for  62%  of  the  variance  in  scores.  Following  varimax 
rotation  with  Kaiser  normalization,  factor  loadings  were  examined. 
Factor  #1  contained  four  of  the  nine  items  in  the  subscale,  Nurse 
Participation  in  Hospital  Affairs;  four  of  the  remaining  five  loaded  on 
Factor  #3.  Six  of  the  nine  items  from  the  Nursing  Foundations  for 
Quality  Care  Five  subscale  loaded  on  Factor  #6;  two  loaded  on 
Factor  #7.  The  subscale,  Head  Nurse  Ability  is  well  represented  by 
Factor  #2,  with  the  addition  of  the  item  “Head  nurse  consults  daily 
with  staff...”  (loaded  at  .804).  The  items  from  the  subscale,  Staffing 
and  Resource  Adequacy  all  loaded  on  Factor  #5  with  loadings  in  the 
range  from  .617  to  .819.  The  items  from  the  subscale  Nurse- 
Physician  Collaboration  all  loaded  on  Factor  5  with  loadings  of  .762 
to  .787.  Whereas  the  subscales,  Nursing  Participating  in  Hospital 
Affairs  and  Nursing  Foundations  for  Quality  Care  are  less  distinct 
than  the  other  subscales,  the  instrument  does  measure  distinct 
components  of  the  nursing  practice  environment. 

This  procedure  was  repeated  in  the  2004  data  set.  Five  factors  with 
Eigenvalues  greater  than  1  were  extracted  and  accounted  for  56% 
of  the  variance  in  scores.  Factor  #1  contained  7  of  the  9  items  in  the 
subscale,  Nurse  Participation  in  Hospital  Affairs.  The  two  that  did 
not  load  on  Factor  1,  actually  loaded  on  Factor  2,  along  with  the  4 
items  that  comprise  the  subscale,  Head  Nurse  Ability,  Leadership 
and  Support.  Factor  3  contained  8  of  the  9  items  that  comprise  the 
subscale,  Nursing  Foundations  for  Quality  Care.  The  one  item  that 
did  not  load  on  Factor  3  (active  continuing  education  program), 
loaded  on  Factor  #1,  Nurse  Participation  in  Hospital  Affairs  with  a 
value  of  .650.  Factor  #4  contained  the  subscale,  Nurse-Physician 
Collaboration,  with  loadings  from  .714  to  .740.  Finally  Factor  #5 
contained  the  subscale,  Staffing  and  Resource  Adequacy  (with 
loadings  from  .544  to  .804). 

Overall,  the  subscales  performed  better  in  the  2004  sample, 
possibly  because  the  increase  in  number  available  for  analysis.  The 
subscales  are  clearly  measuring  distinct  concepts  within  the  work 
environment.  The  one  that  is  the  least  distinct  than  the  others  is 
Nurse  Participation  in  Hospital  Affairs.  However,  in  the  2004 
sample,  the  majority  of  the  items  (minus  two)  loaded  together  on  a 
single  factor.  This  provides  evidence  of  construct  validity  of  the 
PES-NWI. 
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Appendix  D 

Research  Categorization  Using  TSNRP  Areas  of  Research 

Identify  the  main  research  priority  investigated  in  this  research  study. 

Please  check  one  item  for  Primary  (Required)  and  one  item  for  Secondary  Priority  Areas  (if 
appropriate) 

Primary  Research  Priority  Area:  (Required) 

_ Military  Deployment  Health 

X_  Translating  Knowledge  &  Research  Findings  into  Practice  in  a  Military  Context 
_ Evidence  Based  Practice 

_ Recruitment  &  Retention  of  the  Military  Nursing  Workforce 

_ Developing  &  Sustaining  Military  Nursing  Competencies 

Secondary  Research  Priority  Area: 

_ Military  Deployment  Health 

_ Translating  Knowledge  &  Research  Findings  into  Practice  in  a  Military 

_ Evidence  Based  Practice 

_ Recruitment  &  Retention  of  the  Military  Nursing  Workforce 

_ Developing  &  Sustaining  Military  Nursing  Competencies 


Other  (fill  in):  Clinical  Resource  Management 

Identify  3-5  key  words  relating  to  the  proposal.  (Required) 

(You  MUST  use  the  CRISP  Thesaurus  for  key  words.  The  thesaurus  is  on  the  web  at: 
http://crisp.cit.nih.gov/crisp/crisp  help.help 


1. 

Nursing  databases 

2. 

Adverse  events 

3. 

Nurse  staffing 

4. 

Nursing  practice  environment 
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Appendix  E 

In-press  Articles  &  Presentations 

Do  you  have  any  articles  or  presentations  ‘in  press’  E^yes  D  no 

If  yes,  provide  copies  and  all  PAO  clearance  information.  All  citations  listed  must  be  in 
APA  format. 
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Appendix  F 

Public  Affairs  Office  Clearances 
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Appendix  G 
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